arXiv:1505.01047vl [math.RT] 5 May 2015 


Representations of affine superalgebras and mock theta 

functions III 


Victor G. Kac* and Minoru Wakimoto^ 


Dedicated to Jean-Pierre Serre on his 90th birthday. 

Abstract 

We study modular invariance of normalized supercharacters of tame integrable modules over 
an affine Lie superalgebra, associated to an arbitrary basic Lie superalgebra g. For this we 
develop a several step modihcation process of multivariable mock theta functions, where at 
each step a Zwegers’ type “modifier” is used. We show that the span of the resulting modified 
normalized supercharacters is 5'L2(Z)-invariant, with the transformation matrix equal, in the 
case the Killing form on g is non-degenerate, to that for the subalgebra g' of g, orthogonal to 
a maximal isotropic set of roots of g. 

0 Introduction 

This paper is the third in the series of our papers on modular invariance of modified normalized 
characters of irreducible highest weight representations L(A) over an affine Lie superalgebra g, 
associated to a simple hnite-dimensional Lie super algebra g. 

We assume that the Lie superalgebra g is basic, i.e. it is endowed with a non-degenerate 
invariant supersymmetric bilinear form (.|.) and its even part gg is a reductive subalgebra. 
(These properties hold if the Killing form k on g is non-degenerate.) The associated affine Lie 
superalgebra is g = g[t, t~^ © CK © Cd, where KT is a central element, g[f, © CK is a central 
extension of the loop algebra g[f,t“^]: 

[at™, = [a, + mdm,-n{a\b)K, a,b ^ g, m,n ^ Z, 

and d = 

Choosing a Cartan subalgebra f) of gg, one defines the Cartan subalgebra of g: 

i] = Cd+t) + CK. 
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The restriction of the bilinear form (.|.) to f] is symmetric non-degenerate, and one extends it 
from to f), letting 


{t)\CK + Cd) = 0, {K\K) = {d\d) = 0, {K\d) = l. 

One identifies with i) via this bilinear form. Traditionally, the elements of corresponding 
to K and d are denoted by 6 and Aq, respectively. One uses the following coordinates on f): 


eq:0.1 


( 0 . 1 ) 


h = 27rz(—rd + z + tK) =: (r, z, t), where T,t £ C, z G 1). 


Given a set of simple roots 11 = {ao,ai, ... ,a£} of q and A S 1)*, one defines the highest 
weight module A (A) over g as the irreducible module, which admits a non-zero vector vjs_, such 
that 

hvA = A{h)vA for h £ 1 ) and QaiVA = 0 for i = 0,... ,i, 


where 0 q,. denotes the root subspace of 0 , attached to the simple root ccj. Since K is a central 
element of 0 , it is represented by a scalar k = A(A'), called the level of A(A) (or A). 

The character ch"*" and the supercharacter ch~ of A(A) are dehned as the following series, 
corresponding to the weight space decomposition of A (A) with respect to 1): 



+'1 _ +„± 2m{-Td+z+tK) 


where (resp. tr~) denotes the trace (resp. supertrace). It is easy to see (as in |K2j . Chapter 
10) that these series converge absolutely in the domain {h £ 1)| Reaj(/i) > 0, i = 0,1,..., .^} to 
holomorphic functions. In all examples these functions extend to meromorphic functions in the 
domain 


eq:0.2 


( 0 . 2 ) 


X = |/i £ f)| Re{K\h) > o| = {(r, z, t)| Imr > 0} . 


Note that, as a 0 [At~^] 0 CAT-module, A(A) remains irreducible and it is unchanged if we 
replace A by A-l-ad, a £ C, and the character of the 0 -module gets multiplied by Throughout 
the paper q = = e~^. 

In the case when 0 is a simple Lie algebra, there exists an important collection of integrable 
0 -modules A (A), whose normalized characters have modular invariance property |K2] . Recall 
that the normalized (super) character ch^ is defined as 


eq:0.3 


(0.3) 


ch^{T,z,t) = q"^^ch.^^j^^{T,z,t), 


where is given by the following formula; 


|A -I- sdim 0 

~ 2{k + h'^) ^ 


where p is the affine Weyl vector, dehned by 2(p|aj) = (ai\ai) for all a* £ 11 , and hA is its 
level, called the dual Coxeter number. Throughout the paper we assume that the level of A is 
non-critical, i.e. k + hA ^ Q. Note that ch^_|_^j-^ = ch^, a £ C. 

Recall the action of S'A 2 (M) in the domain X in coordinates (jO.ip : 


eq:0.4 


(0.4) 


b \ , f ar + b z c(z| 2 :) \ 

d J ’ ’ \cT-|-(i’ cT-l-d’ 2(cr0d)y 
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By definition, modular invariance of a finite family of functions on X means that the C-span 
of this family is 5 L 2 (Z)-invariant with respect to the action (jn.4|) . 

If the Killing form «; on g is non-degenerate (which is equivalent to 7 ^ 0) a g-module 
L(A) is called integrable if for any root a of g, such that K{a, a) > 0, the elements from the 
root space go, act locally nilpotently on L{A). If g is a Lie algebra, this property is equivalent 
to integrability, as defined in [K2| . For the Lie superalgebras g with h'^ = 0 the definition of 
integrability is similar. However, for the sake of simplicity, we shall restrict the discussion in 
the most of the introduction to the case of non-degenerate k. 

Recall that the defect of the Lie superalgebra g is the maximal number d of linearly inde¬ 
pendent pairwise orthogonal isotropic roots of g. An integrable g- module L(A) is called tame 
if the set of simple roots H contains a d-element set T, consisting of d pairwise orthogonal 
isotropic roots, orthogonal to A. The (super)character formula for a tame integrable g-module 
L(A), conjectured in |KW3| and proved in [GKj (for “good” choices of T) reads: 


eq:0.5 (0.5) 


eq:0.8 




o^+P 


o&W# 


ri/SsT (lie d) 


Here is the affine (super)denominator: 


= eP 


n 


aSAn 


( 1 -e- 


n 


oeAj 


( 1 ± e-“)’ 


Ao^+ and Aj _,_ are the sets of positive even and odd roots of g (counting multiplicities); W = 
IF IX ti, is the affine Weyl group , where IF is the Weyl group of gg, and the subgroup ti consists 
of translations t^, j G L, where L is the coroot lattice of gg, which are defined by 


eq:0.6 (0.6) 


^(A) = A + A(K )7 - ((A| 7 ) + A G ^; 


VP# = VF^ X (resp. IF^) is the subgroup of the affine (resp. finite) Weyl group, generated 
by reflections in a G A+ (resp. a G A+) with K{a,a) > 0, where is the sublattice of L, 
spanned by the coroots a with «;(q:, a) > 0 of gg; finally, £^{w) = (—for a decomposition 
of tc in a product of s+ reflections, with respect to non-isotropic even roots, and s_ is the number 
of those of them, for which the half is not a root. 

Note that formula (|0.5I) for the supercharacter can be rewritten, after multiplying both sides 
by a suitable power of 5 , as 


eq:0.7 (0.7) 


sdimg ^ 

R~ chX = 


•]&W* 


■(w^)w^(0i+p,T,e-)> 


where ©A+pTe- ^ (signed) mock theta function of degree k + hA. 

Recall that for A G f)*, such that X{K) > 0, a (signed) moek theta function of degree 

n = \{K) and defect d= |r|, attached to a lattice Q, is defined by the following series |KW 6 j . 

[KM]: 

(0.8) = q 2 n e( 7 )t^ 

76 Q 


n/3eT(l — e 
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where Q C f) is a positive definite integral lattice, e : Q —)• {±1} is a homomorphism, are the 
translations, defined by (10.61) . and T C f) is a hnite subset, consisting of pairwise orthogonal 
isotropic vectors, orthogonal to A. This series converges to a meromorphic function in the 
domain X, which in coordinates (10.11) takes the form 


eq:0.9 


(0.9) 




E 


^(7- z) 


^ni2^^27rm7(2) 


n^n/3eT(l - 


Of course, if T = 0, we get the usual (signed) Jacobi form, which is contained in a hnite modular 
invariant family (up to a weight factor). 

sdimg — 

The normalized superdenominator R~ is modular invariant (up to the same weight 
factor), since it can be expressed as a ratio of products of the standard Jacobi form ■dn of degree 
2 and powers of the ry-function, see (j4.5p . Therefore, modular invariance of supercharacters 
reduces to that of numerators, i.e. the RHS of (10.71) . 

sdim g — . 

Note that the normalized denominator q 24 R+ is expressed in terms of the Jacobi forms 
■dll and -dio, and the powers of the Ty-functions, see (14.6p . hence it is not quite modular invariant, 
but is a member of a modular invariant family of three functions |KW 6 ] . Likewise, in general, if 
a family of super characters is modular invariant, then the family of characters can be included 
in a modular invariant family (see subsection 6 . 6 ). For that reason, in the remainder of the 
paper we discuss super characters, rather than characters. 

If 0 is a simple Lie algebra, or a defect 0 Lie superalgebra (= osp(l|n)), and L(A) is 
an integrable 0 -module, then formula (jO.Sp turns into the usual Weyl-Kac character formula, 
where = W, and T = 0 (in this case, of course, ch"'' = ch“ and e+(tc) = £-{w) = det(r(;)). 
Therefore (10.7|) holds with the usual Jacobi forms, which easily implies that the finite set of 
normalized characters of integrable 0 -modules of given level is modular invariant, see [K2] . 
Chapter 13. 

However, modular invariance fails for mock theta functions, but sometimes it can be achieved 
by adding non-holomorphic real analytic corrections, discovered by Zwegers [Z] . 

The key role in the modihcation procedure of [Z] and our papers [KW 6 ] . [KW7] . has been 
played by the following rank 1 mock theta functions (m G Z>o, s G Z): 


eqO.lO (0.10) 


$[™’^l(r,zi,Z2) = 


27Timn{zi-\-Z2)+27Tiszi +sn 

E— 


2 _ ^2'Kiziqn 


In order to make these functions modular invariant one introduces the following modifier, see 
[Z], |KW 6 j and (I1.14h of the present paper: 


eqO.ll 


( 0 . 11 ) 


Zl,Z2) = ^hm(T, Qj,m{T, Z 1 +Z 2 ), 

j£s-\-Z 

s<j<s-\-2m 


where z) are rank one Jacobi forms, given by (II2D, and Rj^m{T,z) are certain real 

analytic, but not holomorphic, functions, given by (|1.12l) . see [Z], [KW7| . Then the modified 
mock theta functions 


eqO.12 


( 0 . 12 ) 


g Z>o,s G Z, 
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satisfy beautiful modular and elliptic transformation properties, described by Theorem II.31 
cf. [Z], |KW6j . |KW7j . In particular, it follows from Theorem 11.31 that the function is 

independent of s G Z, hence it is denoted by G Z>o. 

As we have shown in |KW6| . |KW7| . the functions appear very naturally in repres- 

netation theory of g, where g = si{2\l), the simplest basic Lie superalgebra of non-zero defect. 
In this case, choosing both simple roots ai, a 2 of s£(2|l) isotropic with (a!i|a 2 ) = 1, and taking 
T = {ai}, the weights 


Am;s = {171 — s)Ao + sai, where m,s£ Z>o, 0 < s < m. 


are highest weights of tame integrable g-modules of level m, and formula (10.71) for superchar¬ 
acters reads: 


eqO.13 


(0.13) {R-chl^,J{r,zuZ2,t) = zi, Z 2 ) - -^ 2 ,-zi)) 


where 


R {T,Zl,Z2,t) 


^g27^^^ ilirf’&iijT, zi + Z 2 ) 

i9n(T,zi)i9n(T,Z2) ’ 


and, in coordinates , z = - 2:102 - Z2ai. 

Consequently, the modified supercharacter = ch^^.^, obtained by replacing in the 

RHS of (I0.13P the function by its modification is independent of s and 

is modular invariant. Thus after the modification, the non-modular invariant family of m -|- 1 
supercharacters {ch)(^ ^| 0 < s < m} turns in one modular invariant modified supercharacter 


= (r, 21,2:2,0) ^ 2:1,22) - -2:2,-21)) . 

The only basic Lie superalgebras of rank 2 and non-zero defect are s.f(2|l) and osp(3|2) 
(both have defect 1). The essential difference between these two cases is that in the first case 
the function S-{ta) is identically 1 for a G L^, while in the second case it is not. As we have 
shown in |KW7] . in order to treat the second case along the same lines as the first one, one 
needs to consider the following rank 1 signed mock theta functions (m G s G ^Z) 


eq0.14 (0.14) 


$-[-^^l(r,2l,22) = ^^(-1)' 


^27rimn{zi -\-Z2)-\-27riszi ^mn^-\-sn 




\ _ ^2'Kiziqn 


and the signed analogues Rj ^{t, 2 ) of the real analytic functions Rj^m, given by (I1.12p . Then the 

signed modifier is defined by a formula, similar to (10.111) . where the Rj,m are replaced by 

the Rjjji and the Jacobi forms 0j,m by signed Jacobi-forms defined by (11.81) : the modified 

signed mock theta functions are defined by (j0.12l) . where ‘h is replaced by 4>“. The 

functions satisfy beautiful modular and elliptic transformation properties, described by 

Theorem ll.51 In particular, it follows from Theorem 1 1.5 1 that the functions and <1>“[”^’* 1 

are equal if s — s' G Z. 

The above two modifications are called type A and type B modifications. 

The main goal of the present paper is to obtain similar results for arbitrary basic Lie 
superalgebra g with non-zero defect, different from ps.f(n|n). (The case g = psi{2\2) was treated 
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in |KW 6 | .i For this we develop in Section 3 a d-step modification procedure of a mock (resp. 
signed mock) theta function of defect d, attached to a lattice of rank i. At each step we use a 
modification of type A (resp. B), the end result being a product of a theta (resp. signed theta) 
function, attached to a lattice of rank i — and a product of d functions of the form 
(resp. cj)-[™;s]). 

This modification procedure is used in Sections 5 and 6 to construct finite families of tame 
integrable g-modules, whose modified normalized supercharacters form a modular invariant 
family. Namely, we choose a set of simple roots If of g and a d-element isotropic subset T C If 
if g 7^ osp (2m|2n) with m > n + 2 (resp. two d-element isotropic subsets T, T' C If otherwise), 
such that the set of all level k integrable g-modules L{A) with (A|T) = 0 (resp. (A|T) = 0 
or (A|r') = 0) has the property that the modified normalized supercharacters form a modular 
invariant family. It turns out that these families are naturally parameterized by dominant 
integral level k weights of the Lie superalgebra g', where g' is isomorphic to the defect 0 basic 
superalgebra of g, orthogonal to T. Moreover, it turns out that the transformation matrix 
for the action of 5 L 2 (Z) on these modified normalized supercharacters coincides with the well 
known matrices |K2| for the action of 5 'L 2 (Z) on normalized supercharacters of integrable level 
k modules over g\ 

At the end of the paper we treat the case of basic g 7 ^ psi{n\n) with h'^ = 0 , namely 
g = osp (2n + 2|2n) and D(2,l;a), and the case of “subprincipal” integrable tame osp(3|2)- 
modules. We also construct explicitly all level 1 integrable osp (M| A^)-modules, which allows 
us to establish modular invariance of their normalized supercharacters without modification. 

The results of this paper were reported at a spring school in Bonn in March 2015. 


1 Mock theta functions 


Fix an Adimensional vector space f) over C with a non-degenerate symmetric bilinear form 
(. I .). Fix a free abelian subgroup L in f) of rank m > 0, such that the restriction of the bilinear 
form (. I .) to L is real valued positive definite. 

Let f} be the direct sum of f) and the 2-dimensional space CK 0 Cd, with the symmetric 
bilinear form (. | .) extended from f) by 


(fllCAreCd) = 0, (K\K) = 0, (d\d}=0, (K\d) = 1. 

Since the bilinear form (. | .) is non-degenerate on f) and f}, using this form we may identify f) 
with f)*, so that f) is identified with f}*. This form induces a bilinear form on f}*, also denoted by 
(. I .). The elements, corresponding to K and d in f)* under this identification are traditionally 
denoted by 6 and Aq. 

For each 7 £ t) = 1)* define the translation t^ £ End 1)* by the formula 


eql. 1 


( 1 . 1 ) 


^(A) = A + \{Kh - (A(7) + ^A(iL)|7|') <5, A £ ^. 


Here and further I 7 P stands for ( 7 I 7 ). It is easy to see that t^t^ = for / 3,7 £ f), and that 
t-y leaves the bilinear form (. | .) on f}* invariant and fixes the element K. 

Let A £ f}* be such that its level k := \{K) is a positive real number. Let T = {/3i,..., f5n} C 
f}* be such that (/3i|/3j) = 0 for all i,j, (A|T) = 0, and n < m . Following |KW4] . dehne the 
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mock theta function of degree k and defect n by the series 


eql. 2 


( 1 . 2 ) 


iAr 

0A,T = g 


'TGL 


n/3eT(l e 


Here and further e^ is a function on f), defined by e^{h) = e^^^\q = e“^, and t^e^ = 

Given a homomorphism e : L i—)■ {±1}, depending only on k, we define the signed mock 
theta function of degree k by the series 


eql. 3 


(1.3) 


lAll 

T,e = g 


©A.T.e = 


^{i)h 

'yeL 


n/3gr(l e ^) 


Both series m and (USD remain unchanged if we replace A by A + aS, a G C. Both series 
converge to a meromorphic function in the domain 


eql .4 


(1.4) 


X = |/i G'^lRe 6{h) > o| . 


One uses the following coordinates on f) : 


eql. 5 


(1.5) h = 2Tii{z + tK — rd), 

where t, r G C, 2 : G f}. In these coordinates, q = X = {(r, 2 , t)| Im r > 0} and (|1.2I) takes 
the form 


eql. 6 


( 1 . 6 ) 


0A,T(r,z,t) = e“ 

■y&L 


q g27ri(A+fc7)(z) 

n^g^(l-(7-OI/3)e-2-*/3G)) ’ 


and similarly for 0A,T,e- Here and further A denotes the projection of A on f}*. 

Note that 0 a := 0a ,0 and 0| := 0A,0,e are ordinary theta functions and signed theta 
functions of degree k. Both converge to holomorphic functions in X. 

Sometimes we shall write ®at£ ™ order to emphasize the dependence on the 

lattice L. 

We shall also need the following theta functions and signed theta function, associated to a 
lattice L of rank 1 (where we put t = 0): 


eql. 7 


eql. 8 


(1.7) 



tiGZ 

(1.8) 

©tmCE^) = J^(±i) 


^2nimz{n+^) ^ 

n ^2irimz{n+^) ^m{n+^)^ 


m G Z> 0 ) i ^ ) 


, m G 5 Z>o, i G 5 Z. 




Note that 0j,m and depend only on j mod 2m, and depends only on j mod 4m. 
As before and further on we always assume that |g| < 1 (i.e. Im r > 0) in order to guarantee 
the convergence. Note also an unfortunate clash of notation: m in (II.7p - (jl.l4l) has nothing to 
do with m = rankL. 
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Finally, we shall need the following (signed) mock theta functions, associated to a lattice L 
of rank 1 and |T| = 1: 


p2TTimn(zi+Z2)+2-Kiszi ^mn^+sn 

eqF9| (1.9) $['"’^](t,Zi,Z 2) = J]]-^-37777-;;-, meZ>o,sGZ, 


'Y _ ^27viziqn 


(1.10) <h±[™’*l(r,7l,72) = ^^(±1)' 


^27rimn{zi+Z2)+27riszi ^mn'^-\-sn 

_y_ 

2 _ g27r22l qU ’ 


TH G ^ ^ 


It is well known that (signed) theta functions have nice modular and elliptic transformation 
properties (see e.g. Propositions A2 and A3 in the Appendix to jKWTj). This is not the case 
for (signed) mock theta functions. It was Zwegers who first found their modihcations which 
do have these properties [Z]. Following Zwegers, introduce the following (slightly changed) real 
analytic, but not meromorphic, functions for m G Z> 0 ) j £ 


eql.ll (1.11) 


^) — 


^ {sign(n- i -j+ 2m) -E(V'm,n(T,z))} 


e 2m 


-rH-27rm2 


n=j mod 2m 


where E{x) = 2/^6 x G M, and z) = {n - 

We shall also need the following “signed” analogues of these functions, defined for m G 
iZ>o,j G (cf. lEMl): 


eql.l2 


(1-12) 


^ (±l)" 2 m' {sign(n-i - j + 2m) - E(i/>m,n(r,z))} e 2 r ^+2™^ 

1 

7T G 2 ^ 

n=j mod 2m 


Now, following m, [kwt] , dehne the modifier for m G Z> 0 ) s G Z ; 


eql.l3 


(1.13) 




[m;s] 

add 


(T 71,72) 


E 


R 


■j,m 


jez 

s<_j'<s + 2m 



0j,m('r,7i + 72 ), 


and the signed modifier for m G ^'^> 0 , s € : 


eql.l4 


(1.14) 


$ 


±[m;s] 

add 


(r, 71 , 72 ) 


(e 71 + 72 ). 

j€s+Z ^ ^ 

s<_j'<s + 2m 


Define the modified mock theta functions and modified signed mock theta functions: 


2 add 2 add 


The functions and have beautiful modular and elliptic invariance properties, cf. 

[Z], [KWH] . |KW7] . 
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eql.15 


eql.16 


reml.1 


eql.17 


Before stating the results, let us recall the basic definitions on modular transformations. 
The group G = S'L 2 (M) acts in the domain X in coordinates (|1.5jl as follows: 


(1.15) 


a b 
c d 


(r, z, t) = 


ar + h 


t — 


c{z\z) 


^cr + d’ cr + d’ 2(cr + d) 

This action induces the right action of weight w G of G on functions in X: 


(1.16) 



(T,Z,t) 

b 
d 


{cT + d)~ 


a b 
c d 


{T,Z,t) 


(Actually, this is an action of the double cover of G if re € ^ + Z.) We shall drop w in this 
notation if re = 0. 

Remark 1.1. Action (jl.lBjl induces the following action of the standard basis elements e, h, and 
/ of 5^2(C) on the space of holomorphic functions X on X [F G F)'. 


eF 


hF 

fF 


A 

dr 


F, 


w 


n 9 ^ d 


2=1 


dzi 


d 


^ 2=1 


F, 


d {z\z) d 


2 dt 


This action commutes in the following way with the Laplace operator D (defined in the Ap¬ 
pendix to |KW6j i: 

[e,dl]=0, [KD] = -2D, [f,D] = 2TD. 

We thus obtain a representation of the Lie algebra s^ 2 (C) on FD. 

Let A: G C. If T is a function on X, such that 

F{h + aK) = e'^^F{h), hGX, 


we say that F has degree k. In this case we can write: 

F{T,z,t) = e2”^*F(r,z), where F{t,z) = F{t,z,0). 


The space of function of degree A: on A is G-invariant with respect to the right action of (I1.16p . 
which obviously induces the following right action of G of weight w and degree k on functions 
in r and 2 : 


(1.17) 




(cT -I- d) 


Trifccfz 1 2 ) 

-"'e —F 


ar + b 

CT + d’’ 


CT + d J 


The following example is especially important for this and the next sections. 
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exl. 1 


eql.18 


thl.l 


corl. 2 


thl.3 


Example 1.2. Let f) be a 2-dimensional vector space over C with the basis zi,Z 2 and the sym¬ 
metric bilinear form defined by {zi\zi) = 0, i = 1,2, {zi\z 2 ) = 2ziZ2- Let w = 1 and k = m. 
Then the action (11.171) becomes: 


(1.18) 


1, m 


F 


(t ZUZ2) 

b 
d 


2‘Kimcz-^Z2 

{cT + d)~^ e 


F 


ar + b 

CT + d' 


Zl Z2 

CT + d’’ CT + d 


Theorem 1.3. Let m € Z>o and s, si £ Z. 
(a) One has (cf. 


^1 


\ T T T J 




(b) Ifa,b^'L, then 

Zi + or, Z2 + bT) = ^-mab 

(r, Zi + a, Z 2 + b) = (r, zi,Z2). 

In the case s = si. Theorem ll.3l fal coincides with Theorem 1.12(1),(2) in |KW7] . Also, 
Theorem II.Sl bl coincides with Theorem 1.12(3),(4) from |KW7j . Note that notation in |KW7] 
is slightly different: in the present paper we denote from [KW7] by xhe proof 

of Theorem ll.3f al for distinct s and si is along the same lines as that of Theorem [T3Ka) below. 
Theorem II.3r a) implies the following corollary. 

Corollary 1.4. Let m G Z>o and s,si G Z. Then and these functions are 

fixed by the action M.lSil of the group S'L 2 (Z). 

Due to Corollary 11.41 for m G Z>o, s G Z, the function is independent on s, hence we 

shall denote it by 

Theorem 1.5. Let m G ^Z>o. 

(a) If s, Si G Z and s', G ^ Z, then 

\ T T T ) 




1 £1 £2 
? ? 

T T T 

_i £1 £2 

T T T 

^ £1 £2 
T T T 




2‘Kim 

re r 


ZI,Z2), 
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corl .4 


Reml.6 


eql.19 


eql.20 


(b) If s, s' e are such that m + seZ, m + s'G^ + Z, then 

(r + 1 , , Z2) = (r, ^1,^2). 


('cj If s € |Z, and a, 6 € Z have the same parity, or if m € Z>o and a, 6 G Z, t/ien 

+ar,Z 2 + 6 r) = (± 1 )“ zi, ^ 2 ), 

zi + a, Z 2 + 6 ) = 6 ^-““ 21 ,^ 2 ). 

(d) If m is not an integer, s G ^Z, and a,b E 'Z have different parity, then 

+ar,Z 2 + &t) = (±l)“ (r, zi, Z 2 ), 

+ a,Z 2 + 6 ) = e 2 -““$T[™-*l(r,zi,Z 2 ). 

A special case of Theorem 11.51 is Theorem 4.14 in |KW7j for the functions which 

~_f 1 1, 

coincide with the functions <h of the present paper. A proof of Theorem ll.Sl ai is given 

in the next Section. The proof of claims (b), (c) and (d) of this theorem is then straightforward, 
cf. the proof of Theorem 1.11(2),(3) in |KW7| . 

Corollary 1.6. (a) Let m G ^Z>o, and let s,si G ^Z be such that s — si G Z. Then 

(b) For each m G 5 Z>o the four functions where s G 

Z, s' G ^ + Z, are permuted by the action I11.18\} of SL 2 {TI). The functions (resp. 

^-lm-,s']j fixed by 5 L 2 (Z) if m ^ Z>o (resp. G ^ + Z>oJ. If m £ Z>o, then 

fixes and 5 = fixes If m e 5 + Z>o, then fixes 

and S fixes y/jg remaining functions are transposed by these transformations. 


Remark 1.7. Recall the Zwegers’ function pL, given by 


Z 2 ) piir, zi, Z 2 ) = 

nSZ 


q^{n^+n) ^2iTinz2 
_ g27ri2iq’^ 


Note that the RHS coincides with the function 4) ^ 2 ’fi{T,zi,Z 2 ), see (11.1011 . Due to identity 
(5.16) from [KW6| . we deduce the following identity: 

(1.19) i?ii(r, + Z 2 ) ^~'^^'^\t,zi,2z 2 - zi) = I}ii{t,Z 2 ) 4>W(r, zi, 2 : 2 ), 
and from |KW6] . (5.18), we deduce a similar identity for the modified functions: 

(1.20) i?ii(r, + Z 2 ) ^~^^''^\t,zi, 2 z 2 - zi) = I}ii{t,Z 2 ) ^^^\t,zi,Z 2 ). 
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2 Proof of Theorem ll.SIf a^ 


The proof of Theorem ll.5r a) is based on a series of lemmas, which are generalizations of the 
corresponding statements in |KW7| . Since the proofs are similar, we are omitting them. In all 
lemmas of this Section, m S and s, s', si, G 


Iem2.1 


Lemma 2.1. 


(a) If s £2,, then 

(i) ^2nijnm^ 

(ii) Res^i=„+jv$±["'’"]|^ = g2uiinm_ 

(h) If s' £ ^ + 2, then 

(i) ReS,,i=n+j^$+["'’^'l = (-1)'^ ^ ^- 2 -Kijmz 2 ^2-Kijnm^ 

Res = (-1)^+" ^ ^- 2 nijmz 2 ^27Tijnni^ 

(ll) Res 1 5 = (-1)^' ^ ^- 2 -.ijmz 2 ^ 2 .ijnm^ 

ReS,,l=n+j^$-["'’^'l|^ = (-lp+^ ^ ^-2-nijraz2 ^2-Kijnm 


□ 


cor2.2 


Corollary 2.2. Let s,si G 2,s',s'i G ^ + Z. Then the functions _ 

holomorphic in the domain Xq = 

{(r, zi, Z2) G C^l Im r > 0 } . 


Iem2 ■ 2 | Lemma 2.3. 

(a) ^>±[™i'*l(T,-zi,-Z 2 ) = ■ 21 ,- 22 ). 


(b) If a,b £2 have the same parity, one has 


+a,Z2 + 6) = e2-““^>±I™’^l(r,zi,Z2). 


(c) 


lem2.3 


(i) If m is an integer and a,b £ 2, one has 

+ a,Z2 + 6) = e2-““^.±[-’*l(r,zi,Z2). 

(ii) If m is not an integer and a,b £ 2 have different parity, one has 

<^Mm-,s]i^r,zi + a,Z2 + b) = zi, ^ 2 ). 

Lemma 2.4. (a) zi, 2 : 2 ) — zi, 22 + 2t) 

0<fc<2m 

kGZ 


□ 
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(b) ^>±[™;"!(r,zi,Z2) - - 2t,Z2) 


lem2.4 


eq2.1 


eq2.3 


lem2.5 




0<fc<277 

fcez 


□ 


Lemma 2.5. If j € Z, then 

+ jT,Z 2 + jz) = 


□ 


As in |KW6| . [KW7| . consider the following change of coordinates: 
( 2 . 1 ) 
and let 


Zi + Z2 Zi - Z2 

zi = V — u, Z2 = —V — u, i.e. u = --—, V = —-—. 


eq2.2 (2.2) 


^pMm-,s]^T,u,v) := $±["^’"l(r,2i,Z2)- 


Recall the action ()1.18p of 5L(2,Z). Throughout this section we shall denote this action by 
to simplify notation. Note that in the new coordinates we have: 

(2.3) u, v) = , where 5 = . 

Lemma 2.6. 

(a) 

(b) For a,b € Tj one has 

{T,u + a,v + b) = u, v), 


(c) 


(i) If m is an integer, and a,b £ ^Z, such that a + b £ X, one has 

^pMm-,s]^T,u + a,v + b) = {t,u,v). 

(a) If m is not an integer, and a,b £ ^ + Z, one has 

u + a,v + b) = u, v). 


□ 


lem2.6 Lemma 2.7. 
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Iem2.7 


lem2.8A 


lem2.8B 


(a) w, i;) — tt — r, t> — r) 

0<j<2m 

j€Z 


(b) U, v) — e47r*m('»+«)(^±[m-;s](|T-^ ^ — y). 

= E e±,^„,„(r.2,.). 

0<j<2m 

j€Z 


□ 

Lemma 2.8. (r, u, u) — u, ?; — 2r) 

= E e±,^.,„(r, 2 «). 

0<A:<4m 

fcez 

□ 


Lemma 2.9. For s, si € Z, we put 

G++[™;"’"i](t,w,?;) :=(^+['"’"l(r,u,7;) -¥?+[”"’*i]|^(t,7x,7;). 


Then 

(a) G++[™i'*’^i](r,u,z;) - (r, u, u - 2r) 

= ^ e\„(r, 2 u). 

k^X 

s</c<s+4m 


("6; G++[™;"’"i] (r,«, u + 2) - (r, u, v) 


-I , . ._i 

:(-zt) 2 


\/2m 


E E 

j^’L/2m'L 


TTijk 2Trim 

g m g T 




k^X 

Si <fc<S]^ +4m 


("cj is i/ie unique holomorphic function in the domain Xq, satisfying conditions 

(a) and (b). 


□ 


Lemma 2.10. For s G Z, s' G ^ + Z, we put 


Then 
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Iem2.8C 


lem2.9 


(a) (r, u, v) - (r, u,v- 2t) 




fcez 

s<fe<s+4m 


(b) (r, u,v+ 2)- (r, u, u) 

Vzm ^ 


fce^+^ j^'Ll2m'L 
^ <fc<s^ + 4m 


(^cj G 1 is the unique holomorphic function in the domain Xq, satisfying conditions (a) 

and (b). 


□ 


Lemma 2.11. For s', G ^ + Z, we put 


^—[m-,s',s[] u, v) - | 5 (r, u, v). 


Then 
(a) G 


- \m:s ,s‘ 




E ^"’“'9^= &Zt,Jr,2u). 


fc G ^ 

s' <k<s' +411 




y/2m 


(-*^) ^ 


fce^+z j£Zl2mZ 

<fe<S^ +4771 


nijk 2777771/'^. I fc \2 ^ 

erne -r ( + 2 m) 0 (t, 2 u). 


(c) G is t/ie unique holomorphic function in the domain Xq, satisfying (a) and (b). 


□ 


Lemma 2.12. Let m G and j G ^Z. Let v = ar — b, where a,b £ 
(a) If j G Z, then 


; jy „-iiTma^y SX 


= ^i\l - e 
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Iem2.10 


lem2.11 


M + 


d , ^d\li j^- /_1 V 


m 


—IT) 2e T 


J,m\ T ’ T 


= 4:1 g-4nma^y 




k&,l2vriL 


(b) If j & ^ then 






= 4iJI 


^ 1 2'Kim ^.2 , 1 


'ijk 




= 4ix g-4™a2;; 




k£7jl2m7j 


Lemma 2.13. Let j S ^Z. T/ien 

(b) rY{t,v- t) = 2^) {i2j^(r,?;) - i.e. 

(^) Rlmi^.v - t) = 

Rj,mi^^^) + e2^™(2«—)i2-^(r,?; - r) = 2g-i^e2"iE 

fc; ?;) - ?; - 2r) = 2 | ± 


□ 


(-i, f) = - 2 e-^("+ 2 )^ ± e^("+^)=} 


□ 


Let 




feGs+Z 

s<fc<s+2m 


Lemma 2.14. 

(a) If a,b ^ Z, then 


Mm-ts], , , ,x 2Tris(b—a) ±[m;s]/ n 

‘^add + + = e ^ Vadd V^u,v). 
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Iem2.12 


lem2.13A 


lem2.14A 


(b) If m is an integer, and a,b ^ are such that a + 6 € Z, then 

±[™;'5]/ I I 2nis(b—a) ±[m,;s]/ n 

I’M +a,v + b) = e ^ ^add 

(c) If m is not an integer, and a,b ^ 5 + ^! then 

±[m;s]/ I I ,S 2TTis(b—a) =F["i;s]/ \ 

V'add \T,U + a,V + h)=e ^ ^add 


□ 


Lemma 2.15. 

ft!r'ir,u,v) - - 2r) = 

Next, as in |KW7] . in order to study the functions 


E 


e%jT,2u}. 


A:Gs+Z 

s<fc<s+2m 


□ 


^++[m;s,si] _ +[m;s] _ +[m;si]| 

'-^add ■— T^add V^add I S’ 


where s, si G Z, we consider the functions 

:= -R+^{t,v) - -^{-iry^e" 


y/2m 


E 


kGZ 

Si <fc<si +2rj 




Lemma 2.16 

+ +[si] 


(a) u) — — 2t) =—2 + g ^ 2 m{j+ 2 m)v^^ ^ 

(b) a++["'](T,u + 2 )-a++["''(r,u) 


2i 




■jk [" 27viTn I k ^2 27rim | fc + 2m ^2 


k^Z 
Si <fc<si +2rj 


\/2m 

(c) u) zs holomorphic in v. 


_ mjK I ZTTim / I Ag \ 

C m <g T ^ ~'”2m'^ +6 




□ 


Lemma 2.17. 

(a) 

■,++[m;s,si] . 


G*JI"-‘''Hr,u,v) = i Yi »r'*‘'G0eij,„(r,2„). 


3 ex 

s<_j'<s + 2r) 


(b) ^ zs holomorphic in the domain Xq. 
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□ 


lem2.15A Lemma 2.18. 






s<_j'<s+4m 




(r, u, t> + 2) - Cijl"’’’’"' (t, 11, ii) 


\/2m 


(-ir)- 




fcGZ 7GZ/2mZ 
Si <fc<s +4m 


□ 


By Lemma 12.91 and Lemma 12.181 we have 
prop2.1A Proposition 2.19. 


cor2.lA 


’-^add “ 


Then, putting := we obtain 

Corollary 2.20. 

(a) = ^+["*’^ 111 ^, i.e. 

(b) ^+[”^’^1 does not depend on the choice of s €Z. 


□ 


□ 


The remaining cases are treated in a similar fashion. For s G Z, s' G ^ + Z, j G Z, we put 

'-"add ■” ‘^add “Fadd I S’ 


- + M 


(t,u) :=e 'Ti?+^ (- 1 , ^) . 

s^<fc<s^ + 2m. 


Trijk 


lem2.13B 


Lemma 2.21. 


(a) a - u) — “ 2 r) = —2 -j (7 — (7 4 m ^ g27n(j+2m)i 

(b) \t,v + 2)- ' (r, v) 


2i 


y/2m 


(-ir)-i Yi { 


Tvijk f 2Tvim 1 k ^2 27Tim ( \ k-\-2m \2 

m < P T ^ 2m ' -|- Q. T ^ ' 2m ■' 


s^<fc<s^ + 2 m 


} 
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(r, v) is holomorphic in v. 


Iem2.14B 


lem2.15B 


prop2.IB 


cor2.IB 



□ 


Lemma 2.22. 

(a) 


^ — \-[m-,s,s'] 
'-^add 


(r, u, v) 


jez 

s<_j'<s + 2m 


(^) ^add^™’*’^ ^ holomorphic in the domain Xq. 

Lemma 2.23. 

(^) ir, u, v) - (r, - 2t) 

= - Y 0:fc,,„(r,27x). 

k£Z 

s<fc<s+4Tn 


G-add i^^u,v + 2)-G^J '{t,u,v) 


7 ^(—)- E E e^e^(^+^)^ 0 T^(r, 2 u). 

^ fce^+z jSZ/2mZ 

<fc<s^ + 4m 


□ 


□ 

By Lemma 12.101 and Lemma 12.231 we have 

Proposition 2.24. ^ = —G □ 

Then, putting 




we obtain 

Corollary 2.25. 

(a) 

(b) ^“["*’^1 (resp. 1 j does not depend on the choise of s & Z (resp. s' € ^ + Z). 


□ 


Finally, for s', s'j^ G i + Z and j G i + Z, we put 


„- [m-,s _ 

"-^add •— V^add 


^add 15’ 


_fc^l ,v 2,xl 27r'im„.2 

a, [ iJ(r,n) :=-i?..^(r,n) - —=(-ir) 2e ^ 

V2m 


E 

fe G ^ ■|"Z 

s^<fc<s^+2m. 


nijk 

e-—R 


k;m 


1 t? 

r T 
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lem2.13C 


lem2.14C 


lem2.15C 


prop2.1C 


Lemma 2.26. 

(a) a- ■ ^^^\t,v — 2t) 


= -2 <! - q~ ^ 2 -Ki{j+ 2 ni)r 


(h) a- (r,-y + 2) - a- (r, v) 


2i 


y/2m 


, . \_i ’rijfc r 

(—^rj 2 e m 


_ Tvijk r 27Tim / i fc \2 27rim / i fc+2m \2 " 

g m < g T ^ 2m ' -|- g r ' 2m •' 


A; G "1"^ 
s^<fc<s^+2m 


(^cj a - (r, t;) is holomorphic in v. 


Lemma 2.27. 

(a) 


G 


-m;5 ,s- 


add 


\t,u,v) = - ^ aj e_j„^iT,2u). 


je^+z 

<j<s' + 2m 


□ 


(b) holomorphic in the domain Xq. 


□ 


Lemma 2.28. 

a' 

^add 




E 

fc G 

s^<fc<s^+4m 




k,m^ 


(^) G'add (g u,v + 2)- (r, w, ^;) 


y/2m 


fcG^+z jES/2mZ 
<fc<s^ +4m 


□ 


By Lemma 12.111 and Lemma 12.281 we have 
Proposition 2.29. = -G"- 


□ 
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Hence, putting 


cor2.1C 


eq3.1 


eq3.2 


eq3.3 


eq3.4 


we obtain 
Corollary 2.30. 

(a) ^-[”*’^'1 = 




:= if 


+<Jr'. 


(b) if does not depend on the ehoiee of s' £ ^ + Z. 


□ 


Translating Corollaries 12.20112.251 [TSOl from (^=*=1”*’^] to via (|2.ip . (12.21) . we obtain 

Theorem ll.5r a) . 


3 Modification procedure 

Unlike theta functions, mock theta functions do not have good modular and elliptic invariance 
properties. In this section we introduce a modification process of a mock theta function Q^ti 

which, using an n step passage from to for some a* G 2 > 0 ) i = 1,..., n = |T|, makes 
the obtained modified mock theta function Q^t product of n functions and a 

theta function. Since the functions a G Z>o, and the theta functions, have good modular 
and elliptic invariance properties, so does the function Qxt- 

Recall that L C I) = I)* is a positive definite lattice of rank m > 0 and that T = 
{/3i,..., fin} C fi = f)* is such that {fififij) = 0 for all i and j, and m > n. We shall assume that 
L has a Z-basis 71 ,..., 7 ^, such that the following two properties hold: 

(3.1) = i = j = 


(3.2) {lillj) e Z, 1 < i < n, i<j<m. 

As usual, here and further, we let 7 '^ = 27 /( 717 ). 

Let A G f)* be of positive level k = A(iL), and assume that (A|/3j) = 0, i = 1, ..., n. Assume, 
in addition, that 

(3.3) ^ | 7 iP G Z>o, (A| 7 i) G Z, i = 1,..., n. 

Introduce the following notation for s = 0,..., n : 

S S 

(3.4) Ls = Z{ 7 i + ^(7il7p)/3p \s + l <i< m}, = {/3s+i,.. •,/?«}, A* = A + y^fiX\ji)fii, 

p=l i=l 

so that To = T, Tn = Lq = L ,Xq = X. Note that the lattice Lg is isomorphic to the lattice 
Z{ 7 s+i, ...,7m}, hence it is positive definite. 
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eq3.5 


eq3.6 


eq3.7 


eq3.8 


eq3.9 


eq3.10 


eq3.11 


It is a straightforward calculation, using only assumption (13.ip . to show that, if n > 1, the 
function &XT written in the following form: 


(3.5) ^ I t 

76^1 






The first modification step consists of replacing $ by <I> in this formula. For this, we need 
assumptions (13.3p . This modification produces the function, which we denote by Due to 

assumptions (13.ip and (13.21) . + 2/3i) G Z, hence we may apply the elliptic transformation 

property, given by Theorem 1 1.3l bi. and Corollary 11.41 to obtain: 

(3.6) = 

Similarly, if n > 2, we may apply the above modification step to the mock theta function 
®Ai Ti obtain the second modified mock theta function (assumptions (|3.3I) still hold for A 
replaced by Ai): 

©St = a +7^) + 72 + (72 l7i)/3i). 

Repeating the same procedure n times, we arrive at the modified mock theta function 


p-i 


(3.7) 


©A.T •— ©a’^ “ ©aO n kp + 7p + yS(7p |7»)A); 

P=1 


2 = 1 


where 0^" is an ordinary theta function. 

A similar modification process can be developed for a signed mock theta function = 
©A T e± following choice of : 

(3.8) ( 7 ) = 

Here, instead of the first formula in (|3.3p we shall assume 

(3.9) /c|7jp G Z>o, i = 1, ...,n. 

Due to this assumption, the exponent in (|3.8p is an integer. 

We have the following “signed” analogue of (|3.5p : 

©St = E 4(7) i7n- n - 0 ^ (r,-/3i,/3i + 7 ^ + r(7|7r + 2/3i)). 

ll/3eTiU e e) 

The “signed” analogue of (13.61) is 

|2.(' 


0t’l’^ = ci,±[fl7iP;(A|7i)] + 


(3.10) 

and that of (13.7p . called the signed modified mock theta funetion, is 


p-i 


(3.11) 


0t4= ©S4± = ©1;"'" (D-/3p + 7; + E(4l70A), 


X,T,ef 


p=l 


2 = 1 


where 0 ^ is a signed theta function with the sign £^( 7 ) = (± 1 )^DP^ ^ 
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ex3.1 


eq3.12 


eq3.13 


eq3.14 


prop3.2 


Example 3.1. Let q be a simple finite-dimensional Lie algebra with an invariant bilinear form 
(. I .), normalized by the condition that (a\a} = 2 for a long root a. Let ^ be a Cartan subalge¬ 
bra of 0 , and let L C fi be the coroot lattice. It is positive definite even. Let { 71 ,..., 7 ^} C L 
be a set of simple coroots; then condition (I3.2jl is satisfied. Let f) = ^ © CP, and extend (. | .) 
from f) to a symmetric non-degenerate bilinear form on t ), also denoted by (. 1 .), letting 

(/3|/3) = 0, (/ 3 | 7 j) = -Sii, i = 1,..., m. 

Given A € f)* of positive integer level k{= X{K)), and such that (A|/3) = 0, (A|L) C Z, we may 
consider the mock theta function 0 a,{/ 3 }, defined by (jl. 2 p . of degree k and atypicality 1. Since 
conditions (13.3p hold, we have its (1-step) modification (we use here Corollary 11.41) : 

0A,{/5} = 0A^$t^'^^''k©-/3,/3 + 7r)- 

Next we derive modular transformation formulae for the modified mock theta functions 
(13.71) and their signed analogues (j3.1ip . We keep the above notation and the assumptions of 
this Section. In addition, we shall assume that 

(3.12) dimf) = rankL + |T| (=m + n). 

Let M = Ln, M-*- = {/i G f)| {h\M) = 0}, and denote by the orthogonal projection of /i G 1) 
to CM := C ©z M. Fix a positive number k, such that 

(3.13) k{L\L) C Z, 
and let 


(3.14) Leven(resp.L q^^j) = {7 g L| A:( 7|7) G 2Z (resp. G 1 + 2Z)} . 

Introduce the set of weights of level k : 

Pfc = {AGr I X{K) = k, AgM}, 

and let 

Pk,T = {A G Pfc I (A|r) = 0} . 


Proposition 3.2. Suppose that L = Leven- Then for X € P^t we have: 


(a) 


p-i 


®\,TiT,z,t) = ex„iT,z^^\t) 4'[2l>l"l(r, -/3p, Pp + lp +'^ilphi)Pi)- 

p=l 


2=1 


(b) 


0^ ( ^ ^ t 


P{-iT)'''"''\M*/kM\ 2 e z, t); 


mod (fcM+M-*-+C5) 



T + l,Z, 


t) = 


= .irlAP 


0 A,r(c^)^)- 
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prop3.3 


Proposition 3.3. Suppose that 71,...,7„ G -Z^odd- Let ^0 G fl* an element, satisfying the 
following conditions: 


(?ol7i) G ^ {resp. 


■ Zi if G 


(^olT) = 0. 


Then for A G Pk,T we have: 
(a) 


p-i 


eff(r,z,t) = e£'-"(r, 2 (l>,t) f] 4 ,±Ill>p!(^np)l(r, + 7 ; + E(7l7i)ft). 

P=1 


i=l 


and the same formula with A (resp. Xn) replaced by X + (resp. Xn + £,o). 


(b) 




mod (fcM+M-*-+C5) 


q-L 

^\,T 




t^^Pk,T 

mod (fcM+M-L+Ci) 


0 




0 


A+So,r 


-\L 

A+?o,r 


1 2; Izp 

r ’ r’ 2r 


i^ePk,T 

mod {kM + MP+CS) 


P{-iT)^\M*/kM\-2 


i^ePk^T 

mo d (kM+MP+CS) 


(c) 


©a-HctC"^ + 1’ 0- 

In order to prove these propositions, let 


eq3.15 


(3.15) 


min(rx,p—1) 

7p = 7p + (Apl7i)/3i, p = 1,..., m. 

i=i 


Note that M = h{^n+i, ■ ■ ■ ,7m} , and that A^ is the orthogonal projection of A on M and 
Xn{K) = k. We have the following decomposition of 1) in an orthogonal direct sum of subspaces, 
due to assumption (I3.12p : 


eq3.16 


(3.16) 


f) = CM © Pi © ... © Pn, 
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eq3.17 


eq3.18 


lem3.4 


lem3.5 


where 

Vp = C(3p © C7p, p=l, 

Note that the matrix of the restriction of the bilinear form (. | .) to Vp is 
respect to (j3.16l) . we have the decomposition of h S f) : 

(3.17) h = h^^'>+ + ... + h^^\ 


0 -1 
-1 l7p 


2 . With 


Since the modified mock theta function (given bv l3.7h . can be rewritten 


as 


0A , T=n 

P=1 

and similarly for the modihed signed mock theta function O^ T’ since A„ € CM, we can 
write 


(3.18) 


0A,r(C2,i) = ]J$[2l>l"5(r,4^)), 

p=i 


where we put for h € Vp : 

l^pl 

and similarly for the modified signed mock theta functions (I3.11j) . 

Now Proposition 13.21 follows from Theorem 1 1.3f al and the modular transformation formulae 
for theta functions, given, e.g., by Proposition A2 from the Appendix to |KW7| . 

Similarly, the proof of Proposition 13.31 follows from Theorem 11.51 (a), (b), the modular 
transformation properties for signed theta functions, given by Proposition A3 from |KW7j . and 
the following lemma. 

Lemma 3.4. // 71 ,..., 7 ^ G Todd, then we have the following simple formula for ef defined by 

Proof. It is straightforward, using assumption (I3.13p . □ 

In conclusion of this section we derive elliptic transformation formulae for the modified 
mock theta functions and their signed analogues. For each p = 1,... ,n consider the following 
sublattices of the spaces Ip : 

Mp = Z-^/3p + Z 7 p, Mp = Z|7pp/3p + Zjp. 

Lemma 3.5. Let || 7 pP G |Z>o and (A| 7 p) G |Z. 

(a) For V G Mp, the following formulas hold: 
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(i) /i + ti) = /i); 

(ii) /i + rt>) = /i). 

(b) In the case ||7pP S ^> 0 ) following formulas hold for v G Mp : 

(i) $±[||7pP;(>'|7p)](t-^/ j _|_ ^^ = e27ri(A|i;) J±[||7pp;(A|7p)]^^^ 

$=*=[5l7pP;(^l7p)](|^^/j = (|J-l)(/3pk)e-27rifc(/i|i;)^--||'up^±[||7p|2;(A|7p)]|-^^ 

Proof. Note that for u G p = 1,..., n we have: 

2 

\lp\ 


u G Mp iff (/3p|u), (/3p + G 


u G Mp iff {/3p\v), {/3p + ■r-^Tpl'f^) £ ir^'ypW) ^ 

ITpl np\ 


Now the lemma follows from Theorem 11.51 (c), (d). 


□ 


lem3.6 Lemma 3.6. For v G M, z G Me, and A G Pfc one has: 

z + v^t) = 0; 




et+5.(^. - + ™. *) = et+«.{T.i). 


The same formulas hold if we replace 0 (note that in this case the factor ig \f_ 

Proof. The lemma follows from Proposition A4 from |KW7| on elliptic transformations of signed 
theta functions, using that (—1)^1’'!^ = e2’r*(M5ol«) fgj. y g ]\/[^ □ 

Due to Proposition I3.3f al. Lemmas 13.51 and 13.61 give elliptic transformation properties of 
modified signed mock theta functions (13.IIP 


prop3.7 Proposition 3.7. Let A G Pk,T o,nd assume that 71 ,..., 7 n G Todd- Then for v G M' := 
M © M{ © • • • © M(, the following formulas hold: 

(i) 0a-Ko,t(t z + v,t) = z, t); 

(ii) + ru,t) = z, t). 

The same formulas hold if we replace ^0 0 (and remove the factor in (U)). □ 


Similar arguments give elliptic transformation formulae of modified mock theta function 


(I32D. 


prop3.8 Proposition 3.8. In the case L = Teven, the following formulas hold for A G Pk,T o.nd v G 
M := M © Ml © • • • © Mr,: 

(i) Qx,TiT z + v,t) = 0a,t(t t), 

fc L ,|2 ; 


(ii) 0^ j.(r, z + ru, t) = e 27nfc(2|D)^ .^\v\ QL^(^Y^ 2 :,t). 


□ 
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4 Normalized supercharacters of integrable highest weight g-modules 


Let 0 be a basic simple finite-dimensional Lie superalgebra, which is not a Lie algebra. Recall 
that 0 is isomorphic to one of the following Lie superalgebras: si{m + l|n) with m > n > 
1, psi{n\n) with n > 1, osp{m\n) with m > l,n > 2 even, L)(2, l;a), F(4) and G(3) |Klj . 
“Basic” means that the even part 0 o is a reductive subalgebra of 0 and that 0 carries a (unique, 
up to a non-zero factor) non-degenerate invariant bilinear form (. | .), which is supersymmetric, 
i.e. its restriction to 0 q (resp. odd part 0 j) is symmetric (resp. skewsymmetric) and ( 0 ol 0 i) = 0 . 

The subalgebra 0 o, and the dual Coxeter number hy of 0 , introduced below, are given in 
the following table: 


Table 1 


0 

00 


s£{m\n), m > n 

s£{m) 0 s£{n) 0 C 

m — n 

ps£{n\n) 

s£{n) 0 s£{n) 

0 

osp{m\n), m > n + 2 

so{m) 0 sp{n) 

m — 2 — n 

osp{m\n), m < n + 2 

so{m) 0 sp{n) 

|(n 0 2 — m) 

D( 2 ,l;a) 

s£{2) 0 s£{2) 0 s£{2) 

0 

F(4) 

so{7) 0 s£(2) 

3 

G(3) 

G 2 0 s£(2') 

2 


_I4. 

For 0 different from D{2,l]a) and osp{n\n), we denote by 0 g the simple component of 0 o 
of largest rank; in the case of 0 = osp{n\n) we let 0 ^ = sp{n), and in the case 0 = D{2, l;o) 
we let 0 ^^ = s£{2) 0 s£{2), suitably chosen. 

Let 1)^ be a Cartan subalgebra of 0 ^, and let 1) be a Cartan subalgebra of 0 o, containing 
The restriction of the bilinear form (. | .) to and f) is non-degenerate, hence we may 
identify 1 )^ with 1 )^* and t) with i)*, using this form. 

We have the root space decomposition of 0 with respect to f) : 


0 = 1} 0 ( © 0a), 

aSA 


where 0 o = {o S 0 | [h,a] = a{h)a for all h e f)}, and A = {a £ 1)* | a / 0 , 0 a 7 ^ 0 } is the 
set of roots. Unless 0 = ps£{2\2), which we exclude from consideration, all root spaces Qa are 
1-dimensional, and we let p{a) = 0 or 1 £ Z/2Z, depending on whether 0 a lies in 0 q or in 0 j. 
Denote by Ag (resp. Aj ) the set of even (resp. odd) roots, i.e. p{a) = 0 (resp. = 1). The 
Weyl group W C End!)* of 0 is the group, generated by reflections in a £ Ag. 

Let A^ = {a £ Ag I 0 a £ 0 ^} be the set of roots of the Lie algebra 0 ^. Let L C 1} be 
the coroot lattice for the root system i.e. the Z-span of the set {a^| a £ A|^}, and let 
IF^ C End 1)* be the group, generated by reflections in the a £ A^, so that is a subgroup 
of W . We normalize the bilinear form (. | .) by the condition (unless otherwise stated): 


eq4.1 


(4.1) 


{a\a) 


2 for the longest root a in A 


# 

0 • 


Let h'^ be the half of the eigenvalue of the Casimir element of 0 , associated to (. 1.), in the 
adjoint representation. The values of this number, called the dual Coxeter number, are given 
in Table 1. 
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rem4.1 


Remark 4.1. |KW2] Let k{., .) be the Killing form of 0 . It is non-degenerate iff ^ 0, and in 
this case = {a G Ag | K{a,a) > 0}. 

Let T C Aj be a maximal subset of linearly independent isotropic pairwise orthogonal roots 
(all such subsets are VL-conjugate [KW2] 1. The number |r| is called the defect of g. Denote by 
g' the derived subalgebra of the centraliser of T in g. Then 1)' = f] n gms a Cartan subalgebra 
of g\ Let a' C fi'* be the set of roots of g', let W' be the Weyl group of g', and L' C I)' be the 
coroot lattice of the even part of g\ The list of all Lie superalgebras g' and the defects for all 
basic Lie superalgebras g is given in the following table. 


Table 2 


0 

I 

D' 

defect g 

si{m\n), m> n 

sl{m — n) 

m 

osp(m n), m even < n 

sp(n — m) 

m /2 

osp(m n), m odd < n + 1 

osp (I n — m + 1 ) 

(m- l )/2 

osp (m n), m > n + 2 

so(m — n) 

n/2 

osp (n + 2 n) 

0 

n /2 

D( 2 ,l;a) 

0 

1 

F(4) 

s^(3) 

1 

G(3) 

si{2) 

1 


The Lie superalgebra g' has the following properties, which are easily checked by direct verih- 
cation. (For example, for claim (a) we use that the dual Coxeter number for si{n),so{n),sp{n) 
is equal to n, n — 2, ^ 1 respectively |K2| . Chapter 6 .) 


prop4.2 


Proposition 4.2. Let g be a basic finite-dimensional Lie superalgebra, such that g' 7 ^ 0 (i.e. g 
is differnt from osp(n -|- 2|n) and D{2, 1; o )). Then 


(a) The dual Coxeter numbers of g and g' are equal. 

(b) W- = {w£ W*\w{[)-) C f]'}. 

(c) The restriction 0 / (. | .) from g to g' is the normalized invariant bilinear form on g\ 

(d) The coroot lattice L' of g' coincides with Ln f)' and is isomprphic to the lattice M . 

(e) ^ sdim g/f) + |T| = ^ sdim g7fl^ 


□ 


Choose a subset of positive roots A_|_ in A, let A£_|_ = A;, n A+, e = 0 or 1 , and let 
n+ = © go. Let n = {ai, ..., ce^} C A_|_ be the set of simple roots, and let 6 G A_|_ be the 

Q!eA+ 

highest root. Let ps be the half of the sum of the elements from Ae_|_, and let p = pQ — pi- Then 
one has [KW2| : 


eq4.2 (4.2) 


1 h'^ 

2{p\ai) = {ai\ai), i = 1,... ,L, h'^ = {p -6\e); {p\p) = — sdim g 


Recall that sdim g := dimgg — dimgj. 
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Let C a' be the subset of positive roots, corresponding to the linear function on f)*, 
defining A_|_, and let p G f}*' be the element, as defined above, for A:|_. 

Let 0 = 0 [t, © CK © Cd be the affine Lie superalgebra, associated to g. The invariant 

bilinear form (. | .) on g is extended to a non-degenerate invariant supersymmetric bilinear form 
(. I .) on 0 , letting 

{at^\be) = 6m,-n{a\b), {Q[t,t-^]\CK + Cd) = 0, {K\K) = {d\d) = 0,{K\d) = 1. 

Let i) = fi+CAT+Cd be the Cartan subalgebra of g. The restriction of the bilinear form (. | .) 
to f) is non-degenerate and symmetric. Note that this notation exactly corresponds to the one, 
introduced in the beginning of Section 1. In particular, we have 6 and Aq in f)*, corresponding 
to K and d, and we have translations G End!}* for 7 G f), defined by 0. Note also that 
the action of the group W on f)* extends to 1)*, keeping 6 and Aq fixed. 

We have the root space decomposition of g with respect to f) : 

0 = 1} © ( © ga), 0 o = {a G 0 I [h, a] = a{h)a for all h G 1)}, 


where A = {a G (i*| a / 0, g^ / 0} is the set of all affine roots, i.e. the roots of g. Note that 
A = A’’'^ U A™, where 


A’’*^ = {a + n(5| a G A, n G Z}, A™ = {n6\ n G Z, n / 0}, 


the multiplicity of a G A’’'^ being 1, and of n5 being i. Note that p{a + n6) = p{a). 

The affine Weyl group W C End f)* is the group, generated by reffections in the a G Ag®. 
This group has two important homomorphisms : IT —)• {©!}. Namely, given a decomposition 

of tc as a product of reflections in W, s_ of them being with respect roots whose half is not 
a root, we let e^{w) = (— 1 )^±. Recall that W = W k {t7l7 G Z-span of the set of coroots 
, where a G Ag} [K2] . The group W contains the important subgroup 


eq4.3 


(4.3) 


W* = W* V. G L}, 


which is generated by reflections with respect to roots of the form a + n6, where a G Ag 


# 


n 


G Z. 


prop4.3 


Proposition 4.3. 


(a) One has: 


1, 7 G L. 


(b) If Q = si{m\n), psl{n\n), osp{m\n) with m even, L>(2, l;a), or F{4:), then e = on 
W , in particular, s~{t^) = £~^{t-y) = 1, 7 G A. // g = G{3), then e~ = £~^ on , in 
particular, £~{t^) = £~^{t^) = 1, 7 G L. 


(c) If Q = osp{m\n) with m odd, then £ (t^) = (— 1)(t'I7)/2^ 7 g A. 


Proof. Claim (a) is (6.5.10) in [K2j . Claim (b) holds since for all these g the half of an even 
root is never a root, hence s+ = s_. Claim (c) can be deduced by the discussion in §6.5 of 

[K2]. □ 
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The set of postive afhne roots, corresponding to the choice of A_|_, is 
A_|_ = A_|_ U {a + n5\ a € A U {0}, n > 0}, 
its subset of simple roots being 

n = {ao = 5-0}un. 

Let p = hyA-o + p G f)*. One has: p{K) = hA. Define the fundamental weights Aj G f)* by 
(Aj|aJ) = (ijj, i,j = 0, where = 2a/{a\a) (resp. = a) if a is a non-isotropic (resp. 
isotropic) root. (The fundamental weights are defined up to adding a multiple of <5; for Aq we 
take the one we had before.) 

Define the positive subalgebra of g : 


h+ = © ga(= n+ + ^ gt"). 

n>0 


Given A G f)*, the highest weight g—module L(A) is defined as the irreducible g-module, 
admitting an even non-zero vector ua (called the highest weight vector), such that 

h+UA = 0, hvA = A{h)v\ for /i G f]. 

Note that for this choice of 11 the definition of L{A) (as well as of p and p) depends only on the 
choice of the set of simple roots 11 of g. 

The central element K acts as the scalar k = A{K) on L{A), called the level of A (and 
of L(A)); the level k = —hA is called critical. Given 7 G A_|_, the g-module L{A) is called 7 - 
integrable if g_.y acts locally nilpotently on this module. The g-module L(A) is called integrable 

(cf. |KW3] . [GK] i if it is integrable with respect to the subalgebra , that is, if L{A) is 

4^ 44 

7 -integrable for all 7 G Ag and all 7 = a + nJ with a G Ag and n G Z>o. 

Since any integrable highest weight module of level A: = 0 over an affine Lie algebra is 1- 

dimensional |K2j . and since the ideal, generated by the affine Lie subalgebra g^ in the affine Lie 
superalgebra g is the whole g, it follows that any integrable level /c = 0 highest weight g-module 
is 1 -dimensional as well. 

Define the character and supercharacter of the g-module L(A) by the series, corresponding 
to the weight space decomposition of L{A) with respect to f) : 

= trL(A)e'', ch"(^)(/i) = strA(A)e'*, /i G 


The series ch^^^^ converge to holomorphic functions in the domain {/i G f)| ai{h) > 0,i = 
0 ,..., £} C A, and one has: 


eq4.4 (4.4) 


e '^ch+^)(r,z,t) = (e ^ch^(^))(r, z + ^, t). 


where ^ G f) is such that a{^) G Z (resp. G ^ + Z ) if a G Ag (resp. G Aj). 

For integrable L{A) the explicit formulas for ch^^^^, given below, show that these holo¬ 
morphic functions extend to meromophic functions in the domain X (defined by (II.4p ). We 
conjecture that this holds for arbitrary L(A). 
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eq4.5 


eq4.6 


eq4.7 


eq4.8 


eq4.9 


eq4.10 


In order to write down the (super)character formulae, define the normalized affine denomi¬ 
nator and superdenominator R~^ and R~ by the formulae: 


R^ = eP - 


—n5\l 


n. 


:GA 


0 + 


(1 - e-“) 


naeAi+(lie- 


in coordinates (jl.5p they look as follows |KW 6 | : 
(4.5) 


oeA+ 


(4.6) 


R+{T,z,t) = 


n 


aSA, 


0 + 


"dii{T,a{z)) 


Ua&A-,+ ^w{r,a{z))' 


where = lA^+j, and ?9ii, -dio are the standard Jacobi theta functions of level 2 (see Appendix 
of [KW 6 ] b 

Recall that the normalized (super) character of A (A) is defined as 


. |A+p|^ sdim [1 

ch^=qW+f^ 24 

Here and further we assume that the level k is non-critical, i.e. k + hy ^ 0. 

Note that ch^ remains unchanged if we add to A a scalar multiple of 6. In particular, from 
the above discussion it follows that ch^ = 1 if A has level 0. Hence we shall exclude from 
consideration integrable g-modules of level k = 0. 

Recall that for an integrable highest weight g-module A (A), satisfying certain conditions, 
the following (super) character formula holds |KW3| . |GK| : 

, I |A+PP , 

(4.7) R^ch^ = qHk+h'^) ^ £^{w)w 

w£W* 


gA+p 

rwi±7^’ 


where T is a maximal linearly independent subset in Aj_)_, such that 

(4.8) (A + ^T) = 0 and (T|r) = 0. 

Formula (j4.7l) is proved in [GK] . provided that /i^ / 0, under the following two conditions on 
T: 

(4.9) |r|= defect 0 , 


(4.10) Ten, 

for a “good” choice of T C H. 

We conjecture that this is valid (with minor modifications) also for g = osp(n + 2|n) and 
11(2,1; a) (this is proved in [GK| only for vacuum modules). For these g (i.e. different from 
psl{n\n)), condition (14.Op is equivalent to (I3.12p . which is essentially used in Section 3. By 
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eq4.11 


eq4.12 


eq4.13 


eq5.1 


eq5.2 


eq5.3 


(j4.2l) . second formula in (I4.8p and (I4.10p . {p\T) = 0, hence, provided that (|4.10p holds, (|4.8I) is 
equivalent to 

(4.11) (A|r) = 0 and {T\T) = 0. 

We shall exclude from consideration g = ps(!.{n\n) since condition (I3.12p doesn’t hold in this 
case. The case g = psi{2\2) was studied in |KW 6 | . 

Note that, due to (j4.4h . it suffices to discuss the super character formula. We shall express 
the normalized super character ch~^, given by (14.7p . in terms of mock theta functions &XT 
(resp. signed mock theta functions Q^rp^_) in the cases g / osp{m\n) with m < n, m odd, cf. 
Proposition I4.3lf bl (resp. g = osp{m\n) with m < n, m odd, cf. Proposition I4.3f cll. Due to 
(14.31) . we can rewrite (14.71) in the following equivalent form, respectively: 


(4.12) 

R c\i^= ^ e 



and 


(4.13) 



w&W* 


where e ( 7 ) = (— 1 ) 1 ^ 1 ^/^, 7 E L. 

5 Modular invariance of modified normalized supercharacters 

The modified normalized supercharaeter ch^(r, z, t) is defined by replacing in (j4.12p (resp. (j4.13p 
the mock theta function (resp. ©A+pTe-) modification (resp. 

if it exists: 

(5.1) R- ch^ = ^ e-{w)w{Q\^^T), 
and 

(5.2) R- ch^ = ^ 

From the conditions of integrability of T(A) we shall see that in the cases when g osp{m\n) 
with m odd, the conditions for existence of the modification of the mock theta function 
hold. In the case when g = osp{m\n) with m odd, we must modify the signed mock theta 
functions ©a+^oTs ’ which is possible only for the sign function e = given by (13.8p . We shall 

check, using Lemma [TH that in this case one has: £^(^ 7 ) = £”( 7 ) = (—1)1^1^'’^^, 7 G A, hence 
the modification of ©A+pTs- ^^ists, and it is equal to (see (l3.1ip L 

Using (13.181) . we can rewrite (j5.ip as follows: 


(5.3) {R c\i^){T,zfi)= ^ £ {w)w{Qfl^-^)^{T,z^^\t)^k{T,z)), 

w£W* 
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where 


\T\ ^ 

$fc(r,z) = JJ 

p=i 


k + h^ 


l7pl' 


Hr,-Pp{zP), (Pp + r^lp){4^^)), 


hp 


and we use the decomposition (I3.17P of z G i). Similarly, using the signed analogue of (|3.18l) for 
the modified signed mock theta function 0^’^ (given by (I3.1ip h we can rewrite (15.21) as follows: 


eq5.4 (5.4) 


{R chA)iT,z,t)= (r,z)), 


where 

|T| 

P=1 

Since (A + p)n G {k + h'^)Ao + CM + C(5, and CT lies in the orthocomplement to CM in f)*, 
from (15.3|) and (15.41) we obtain the following corollary. 


cor5.1 


Corollary 5.1. The modified normalized supercharacter ch^ of an integrable Q-module L(A) 
depends only on A mod (CT + C6). □ 


rem5.2 


Remark 5.2. Since 71 ,..., 7 ^, fii, ■■■■, fin form a basis of f}, we see that T^.t (defined before Propo- 
sition[32]) lies in /cAq + CT+C(5 + C{ 7 „+i, ..., 7 m}- Since M-*- = CT+C{ 7 i,..., 7 ^}, we conclude 
that Pk,T mod {kM + + C(5) = Pk,T mod {kM + CT + C(5). 


Next, we study modular invariance of modihed normalized super characters of integrable 
0 -modules T(A) with the highest weight A, satisfying conditions (I4.9p - (l4.1ip . For this we shall 
use formula (15.11) (resp. (15.21) 1. obtained from (14.121) (resp. (14.131) by modification. 

Given fc > 0, denote by C f)* the set of level k highest weights of integrable highest 
weight 0 -modules, such that (A|T) = 0, and by fl). C f)'* the set of level k highest weights of 
integrable highest weight J'-modules. Note that flk T Pk,T^ and flji, C flk by extending A G f)'* 
to f)* by 0 on f)'-*- . 


Th5.3 


Theorem 5.3. Let q he one of the basic simple Lie superalgebras si{m + l|n) with m > n > 
1, osp (2n|2m) or osp (2n -|- l|2m) with m > n > 1, osp (2m -|- l|2n) with m > n > 1, T(4) or 
G(3), and let A_|_ C A be a choice of positive roots, such that the following conditions hold: 



(a) the set of simple roots 0 /A+ contains a maximal subset T = {fii,..., fin} of pairwise 
orthogonal isotropic roots, and (fiil'jj) = —dij for some basis 71 ,... , 7 m of L; 

(Hi) for each p G Pk,T, such that iW')^ = {1}, there exists a (unique) w G W\ such that 
w{p) G mod (CT -|-C(5). 


Then the C-span of the set of modified normalized supercharacters of highest weight g-modules 
L{A) with A G Llk mod (CT -|- C6) (see Corollary \5.1\) is SL 2 {Tj)- invariant with the same 
transformation matrix as forg' (which can be found in jKBf, Theorem 3.8(a)). 
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Proof. Let {gi}i^i be a set of left coset representative of W' in First, we consider the case 
0 7 ^ osp (m|re) with m < n + 1, odd. 

By (15.11) and Proposition I4.3l bl we have for A G flfc ; 


eq5.5 (5.5) 


iei w&w- 

By Theorem 4.1 from [KW6j for e = e' = 0, we have: 

eq5.6 (5.6) i?“(— —,t— ■^^) = {—ir) 2 ’’ R~{t, z, t), ii“(r + 1, z, t) = ei2 ®i2“(r, z, t). 

- T T It 


By Proposition 1,3.21 (b) and Bemark 15.21 we have: 

| 2 ' 


P)L 

'^wiA+^,T 


1 Z \Z\ 

T T It 


3q5.7 (5.7) 




,T 

mod ((k + h'^)M+CT+k5) 


Using (j5.6p and (15.7p . we obtain the following modular transformation of (j5.5p 

|2 


chA 1 - 


i&I 




« ^ _ ^ n t 

y s~^{w)e '=+'* 

,T w£W' 

mod {{k + h'^)M+CT+CS) 


,fc+h^ 


If {W')n / {1}, the sum over w G W' in the RHS is zero, hence we may assume that g G 
are such that (VU')^ = {!}• Therefore, applying condition (iii), we may assume that there exists 
a (unique) w' G VF', such that g, = w'{A' + p), where A' G 

Thus, using also Proposition [421(d), the RHS of the above expression for chA (— 7 , f, t — ■^) 
can be rewritten as follows: 

• i (sdim sVnb 1(A: + h'')L' I ■ ^ i?- (r, z, f)-1P(r, z, f), 

where 


i&I 


A'Sfij, mod C5 w,w'(iW' 


Letting w' = w ^w', we obtain: 


B = 




T- 


A'sOj. mod C5 ui"giy' 


i&I 


w'&W' 
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Hence, substituting (15.5p in the RHS of B, we obtain for A G 


eq5.8 (5.8) 


T T 2r 


X 


E 


27ri(A+p|m(A'+p)) ^ 

y e'^{w)e chA'(r, z, t). 


A'Gn^mod CSwGW- 


By assumption (i) we may replace p by p' in this expression, and, by Proposition 02] (a), we also 
may replace hy by hy'. Thus we obtain that ()5.8p is the S'-transformation formula of Theorem 
3.8(a) from |K2] for 0 (A) = g\ For the T-transformation the claim is obvious. 

In the remaining case of g = osp (m|n) with m < n + 1, odd, the proof is similar. The 
difference is in that /i^ G ^ + Z, e~{w) y e'^{w) for w G W' and {p\^i) G ^ + Z (not G Z as in 
all other cases). But then we can take = P-, so that 


^ chA = {Pi) Pi Y, ^ • 

iGl wGW' 

So, the modified mock theta functions involved are of the form ©a-^q t ^ ^ Pk,T^ 
we can use the last 5-transformation formula from Proposition 13.31 claim (b), and the last 
T-transformation formula from claim (c). Note also that in this case Wa_|_^jj = {!}. (In this 
case g' = osp (l|n -|- 1 — m). The Lie super algebra g, where g = osp (l|n), is not considered in 
[K2] . but the proof of Theorem 3.8(a) in this case is the same.) □ 

In the next few subsections we will show that the conditions of Theorem 15.31 hold for some 
choice of A_|_ in g = si(m + l|n) with m > n, osp{2n\2m) with m > n, osp{2m + l|2n), T(4), 
and G(3). 


5.1 Case g = s^(m -|- l|n), m > n > 1. 

As usual, we choose the Cartan subalgebra f) of g to be the subspace of supertraceless matrices 
of the space I) of all diagonal matrices. We choose for (.|.) the supertrace form {a\b) = strab. 
Let El,..., Em+i, Si,... ,Sn be the basis of f), dual to the standard basis of I), so that 

(EjIEj) — Sij, (^SilSj) — Sij, (Ej|(ij) — 0. 

These elements define linear functions on f] via the supertrace form, hence on I) by restriction. 
We have: 


^0 = - ^j\^ <ij <m + l, iy j} U {(5i - 6j\ 1 < i,j < n, i / j}, 

Aj = {±(ei — (5j)| 1 < i < m -|- 1, 1 < j < n}. 

Let 7 i = Em+i — i = 1, ■ ■ ■ ,m, and fii = ei — Si, i = 1,... ,n. Let L be the Z-span of 
vectors 71 , ... , 7 m) and let T = {/3i,...,/3„}. Then L is the coroot lattice (of rank m) of the 
subalgebra g^ , isomorphic to s^{m + 1) of the Lie algebra gg, corresponding to ei,..., Sm+i) and 
the bilinear form (.|.) is normalized as in (|4.ip . Notice that L = Teven and conditions (13.11) and 
(|3.2h hold. Also, condition (13.31) on k holds iff A: is a positive integer, which we shall assume. 
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eq5.9 


Th5.4 


eq5.10 


Furthermore, (/3i|/3j) = 0 for all i,j = 1,... ,n. Recall the set Pk,T, defined before Proposition 
13.21 We clearly have; 

n m 

(5.9) Pk^T = {^Ao + ^ ^2^2 I ^1; • • • ; ^771 ^ C(5. 

2 = 1 2 = 22+1 

Thus, we may consider the mock theta function Q^t construct its modification 

all A G Pfc,T) where k G Z>o. Since condition (I3.12p holds, we have modular invariance of the 

functions 0^^^, given by Proposition 13.21 for A G Pk,T- 

Choose the set of positive roots of q with the subset of simple roots IT = {ai\ i = 1,..., m+n}, 
where: 


022-1 = £i- Si, 02* = Si- Ei+i for 1 < i < n, Oj = - Si-n+i for 2n + 1 < i < m + n, 

so that the first 2n simple roots are odd and the remaining m — n simple roots are even, and 
the highest root is 0 = ei — em+i- We shall consider the highest weight g-modules L{A), 
corresponding to this choice of 11. Recall that h'^ = m + 1 — n and note that 

n m 

p = (m - n) ^ ^ (m + 1 - i)ei. 

2 = 1 2=72+1 

By the same method as in |KW3| . Theorem 2.1, one proves the following result. 

Theorem 5.4. The 'Q-module L{A) with the level k 0) highest weight A, such that (A|T) = 0, 
is integrable if and only if A G Pk^T (see k5.9il ) satisfies 

(5.10) k G Z>o, ki G Z>o, i = 1,.. . ,m, and k > ki > k 2 > ■ ■ ■ > km- 

Proof. Recall that a g-module L{A) is integrable if and only if it is a-integrable for all simple 
roots of g^, cf. |K2j . For the choice of positive roots of g, associated with the above Ft, they 
are: Oj := 0:22-1 + 022 = Si - Ei-i for 1 < i < n, o* = Ei-n - £2-22+1 for 2n + 1 < i < m + n, 
and oo = (5 — £1 + £222+1- Recall also that T(A) is a-integrable for an even simple root a of g iff 
(A|a^) G Z>o. Hence L{A) is Oj-integrable for 2n -|- 1 < i < m -|- n iff (A|ei — £2+1) G Z>o, i.e. 
iff ki — ki+i G Z>o for 2m -|-l<i<m-|-n — 1 and km+n £ ^>o- 

Next, a root Oj is simple in the set of simple roots H^*^ = (H), where is an odd 

reflection with respect to (ii [S], [K^. The highest weight of L{A) with respect to H^*) is A, 
since (A|/lj) = 0. Hence again T(A) is Oj-integrable for 1 < i < n iff /cj — fcj+i G Z_|_ for these i. 

Finally L{K) is oo-integrable iff (A|(i — ei + Em) = k — ki G Z>o . This completes the 
proof. □ 

Note that for A of the form (15.9p we have: 

n m 

A +p = {k+ }i^)AQ-\-''^^^k[fji + k'^Ei, 

2 = 1 2=71+1 

where k[ = ki + m — n for i = 1,... , n, and k[ = ki -\- m -\-1 — i for i = n -|- 1,... , m. Hence for 
integrable L{A) we have by Theorem 15.41 

k + li^ > k!i> ... > k'j^> k'nJ^-i > ... > k'm > 0, k,ki gT,. 
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eq5.ll 


Th5.5 


eq5.12 


Note that (cf. (j3.15p ) 7p = 7p + A- Hence formula (|5.3I) for A of the form (I5.9p . 

such that (jS.lOl) holds, becomes 

w&W* 

(5.11) n ^ p 

n -fiAzP),hp + Eft)(4"’)))- 

p=l i=l 

By Theorem l5.4l the set is non-empty iff A: G Z>0) a-i^d h^ Hris case it consists of elements 
of the form (|5.9p satisfying conditions (15.101) , up to adding multiples of 6 . 

Note that g’ is the subalgebra s£(m-|-l—n) of g^ , corresponding to Sn+i, •••, £m+i- Conditions 
(i) and (ii) of Theorem 15.31 obviously hold. In order to check condition (iii), note that the RHS 
of (j5.1ip is independent of the labels ki,... ,kn of A; if we put all these labels equal k, then, 
by Theorem 15.41 we obtain that mod CT consists of the set of highest weights of all 
integrable highest weight gbmodules of level k. Thus, modular invariance, claimed by Theorem 
15.31 holds in this case. 

5.2 Case g = osp(2n|2m), m > n > 1. 

The set of roots of g is described in terms of an orthogonal basis Si,... ,£m, (^i, • • •, of 1)*, 
such that {£i\£i) = |, (5i|(5j) = —We have: 

^0 = ±i£i + £j), ±2ei\ 1 < i,j <m,i^ j}U {di - dj, ±((5* -4 6j)\ 1 < i,j < n, i j^j}, 

Aj = {±(6, - 6j), ±{£i + (5j)| 1 < A < m, 1 < j < n}. 

Let 7i = —2ej, i = 1,... ,m, and f3i = £i — 6i, i = 1,... ,n. Let L = TL^\ and 

// 

let T = {/3i,..., j3n}. Then L is the coroot lattice of g^ = sp (2m), and the bilinear form is 
normalized as in (j4.ip . Again L = Teven) (^|^) = 0, condition (13.3p on k holds iff A: is a positive 
integer, and is described by (15.9p . Hence we may consider the mock theta function Q^t 

and construct its modification ^ Pk,T- Again, we have modular invariance of these 

modified mock theta functions, by Proposition 13.21 

Choose a subset of positive roots with the subset of simple roots H = {aj| i = 1,..., m -|- 
n), where cXi — £Yn—i-\-i ^m—i for 1 ^ ^ ^ m n, 

£n—1) • • • j CXm+n —3 — ^2 <^2) 2 — ^2 1 — <^1) CXm+n — T SO that 

9 = 2£m. Recall that = m -|- 1 — n, and note that 

m 

p= Y 

2=72+1 

The following theorem is proved by the same method as Theorem 15.41 

Theorem 5.5. The Q-module L{A) with the level k 0) highest weight A, such that (A|T) = 0, 
is integrable if and only if A G Pk,T, described by 115.9\) . satisfies 

(5.12) k G Z>o, ki G Z>o, i = 1,..., m, and k > km > km-i > ... > ki. 


□ 
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Note that jp = 7p for all p, in particular, M = Tj^n+i + • • • + ^7™ is the coroot lattice of 

t i 7 

0' = sp (2m — 2n), the subalgebra of , corresponding to £n+i, Note also that for A of 

the form (15.91) we have 


A + p — (/c + /i^)Ao + kifii + k[£i, 

1=1 i=n-\-l 


where k[ = ki + i — n,i = n + 1,... ,m. Hence for integrable L{A) we have, by Theorem 15.41 
k + > k'^> ... > > kn > kn-i > ■ ■ ■ > ki > 0, k, ki, k[ e Z. 

Formula (15.3p for this A becomes: 


eq5.13 


(5.13) 


{R ch^)(r,z,t)= 

WGW^ 

n 

n -/3„(T>), (/3, + 7,)(4">))). 

p=l 


Conditions (i) and (ii) of Theorem 15.31 obviously hold. In order to check condition (iii), note 
that the RHS of (I5.13P is independent of the coefficients fei, of A; if we put all of them 

equal 0, then, by Theorem 15.41 we obtain that flfc mod CT consists of the set H), of highest 
weights of all integrable highest weight ghmodules of level k. Thus, modular invariance, claimed 
by Theorem 15.31 holds in this case as well. 


5.3 Case g = osp(2n + l|2m), m > n. 

The set of roots of g is described in terms of the same basis as in Section 5.2, but the set of 
roots is larger: one should add roots {(5il 1 < i < n} to Ag (resp. {ej| 1 < i < m} to Aj) from 
Section 5.2. The roots 71 ,... ,7m and the set T = {/3i,... ,/3„} are the same as in Section 5.2, 
and 0Q , L and M are the same. We have: 


eq5.14 


(5.14) 


n m 

Pk,T = {kAo + + X] ^1; •••? ^771 ^ “h 

i=l i=7l-\-l 


In the case m > n (resp. m = n) we choose the following subset of simple roots II of A : 
0^2 — 2+1 ^771—i fol" 1 ^ Til 71 1, CXfyi—Yi — £ 71+1 n+1 — • • • ? Q^m+n—2 — 

62 C^r7i+7i—l — ^1? Q^m+n — (resp. (y.\ — 5n ^711^2 — ^ti—Ii^Z — ^n—1 

6rx-i, ...,a2n-2 = ^2 ” ce2n-i = Si — £1, a 2 n = ^i), SO that the highest root is 6 = 2en (resp. 

— ^71 “h (^n)* 

The conditions of integrability of a highest weight g-module L{A) with A E Pk,Ti given by 
(j5.14p . are (jS.lOp . However, the formula for the modified supercharacter is different since 
and the coefficients of p lie in ^ + Z: 

^ 1 ^ 2 

h}'=m-n+-, p=-'^I3i+ '^{i--)£n+i- 

2=1 2=1 
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For A of the form ()5.14p . satisfying conditions (I5.10p we have 


A + /? — ik + /i^)Ao + k[£i, 

2 = 1 2 = 21+1 


where k[ = ki + ^ for 1 < z < n, and k'- = ki + i — n — ^ for n + 1 < z < m. 
Furthermore, since 2{k + /z'^) is an odd integer, we obtain 


eq5.15 


(5.15) 


7€L, 


Therefore, the modihed normalized supercharacters are expressed in this case in terms of signed 
theta functions and modified signed theta functions. Namely, formula (15.4p for integrable A(A), 
where A is of the form (|5.14p . satisfying conditions (|5.10l) , becomes (due to Corollary 1.5(a)): 


eq5.16 


(5.16) 


{R ch^){T,Z,t)= Y, £ 

n 

p=l 


Here we were able to to replace the second superscript in by ^ due to Corollary 11.61 In 
this case g' is the subalgebra osp{\\2m — 2n) (= 0 if m = n) of g, corresponding to £n+i, •••, Em- 
Conditions of Theorem 15.31 are again easily checked, hence we again have the modular invariance, 
claimed by this theorem. 


5.4 Case g = osp{2m + l|2n), m > rz > 1. 

The set of roots of g is described in terms of an orthogonal basis £i,... ,£m, Si,... ,6n oi 1)*, 
such that (e+i) = 1, ((5+*) = —1. We have: 

+ = 1 < hj <m}U - Sj,±{5i + 5j),±25i\ 1 < i,j <n},i^ j, 

Ai = {ziz{£i — 6j),±{£i + 6j),±6j\ 1 < i < m,l < j < n}. 


Let ji = £n+i — ez for 1 < z < n,^n+i = for n + 2 < z < m, and let 

A = £i — Si, i = 1,... ,n. Then our basic condition (13.ip holds. Let L = Z71 + ... + T^'ym 

II 

and T = {/3i,...,+}. Then L is the coroot lattice of g^ = so{2m + 1), the bilinear form is 
normalized as in (BU), L = Leven j {T\T) = 0, and condition (13.3p on k holds iff A: is a positive 
integer. The set Pk,T is as follows: 


eq5.17 (5.17) = {kAp + kjfSj + ki£i\kiG-'Ij, ki — ki € Z, i = 2, ...,m} + CS. 


2=1 


2=n+l 


Hence we may consider the mock theta function construct its modification 0A,r for 

all A e Pk,T, where k e Z>o, and we have modular invariance of the functions 0 a,T)A € Pk,T, 
given by Proposition 13.21 
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th5.6 


eq5.18 


eq5.19 


Th5.7 


Choose a subset of positive roots in A with the subset of simple roots 11 = {ai = Em — 

E^m—lj • • • ) CUfn—n — n+1 — £71 Ofm—n+2 — Ij • • • j Oim+n—2 — ^2 

£1, ctm+n-i = — Si,am+n = <^i}) SO that 9 = Em + Em-1- Recall that h'^ = 2 {m — n) — 1 and 

m - - 72 

P= {-i-n- -)Ei - 2 Z] 

i=n+l i=l 

The following theorem is proved by the same method as Theorem 15.41 

Theorem 5 . 6 . The Q-module L{A) with the level k 0) highest weight A, such that (A|T) = 0 , 
is integrable if and only if A G Pk,T, described by ^ 5 . 17 \ ), satisfies 

(5.18) k G k-i G —^>Q, k-i k\ G ^>0; km P km—i P • • • P k\^ k ^ km T km—i- 

I jj 

Note that 0' is the subalgebra so{ 2 m + 1 — 2 n) of Qg , corresponding to En+i ,... ,em- R is 
easy to check the conditions of Theorem 15.31 Hence, modular invariance, claimed by Theorem 
15.31 holds in this case. 

5.5 Case g = F( 4 ). 

We consider the set of simple roots H = {ai, a2, 0:3, a^}, where oi is even and a2j«35«4 are 
odd isotropic, with the following non-zero scalar products; 

1 3 

(ai|ai) = 2, (ai|a2) = —1, (Q;2|a3) = 1) (a2|«4) = (a3|<a4) = — 

The highest root is 0 = 2 ai + 3a2 4-03-1- 204. The subalgebra g^ is the simple Lie algebra, 

isomorphic to so(7), with simple roots {02 + 0:3, oi, 02 + «4}- We choose the following basis of 
the coroot lattice L of g^ ; 

71 = oi, 72 = oi -I- 02 + aS) 73 = «i + 202 + 204. 

We let / 3 i = 02,T = Then one has: 

7i = 7i7 72 = 72 + /3i, 73 = 73, P = l 2 + 73- 

Furthermore, A G Pk,T iff, up to adding a multiple of 5 , we have: 

( 5 . 19 ) A =/cAo -4 A:i/ 3 i -4 A:272 + A:373, 

where /c2 — ^1,^1 + k2 — k3,ki — 2k2 — 2k3 G Z; (equivalently: k\ = \{2a2 + 03), k2 = 

|( 3 ai -|- 202 + 0^3), ^3 = |( 3 ai 4-02-1- 203) for some oi, 02,03 G Z). 

We consider the irreducible highest weight g-modules A (A) for the corresponding to H choice 
of positive affine roots. The following theorem is proved along the same lines as Theorem 15.41 

Theorem 5 . 7 . The Q-module L{A) with the highest weight A such that (A|r) = 0 , is integrable 
if and only if A G Pk^T (see h 5 . 19 \) ) satisfies 

k - k2- ks, k2 - ki, ki + k2 - k3, ki - 2k2 - 2k3 G Z>o; 

equivalently, iff for some 00,01,02,03 G Z>o one has: 

A: = 00 + 201-402-403, /ci = i(2o2-403), A;2 =-^(301-4202-403), /cs = i( 3 oi -4 02 -4 203). 

00 O 
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I II 

Next, g' is the simple subalgebra of g^ , isomorphic to s£( 3 ), with simple roots 72,73- Hence 
p' = p, and conditions (i) and (ii) of Theorem 15.31 hold. It is also not difficult to check condition 
(iii). Hence Theorem 15.31 holds in this case as well. 


5.6 Case g = G( 3 ). 

We consider the set of simple roots H = {ai, 02,0:3}, where ai is even and 02, as are odd, with 
the following non-zero scalar products: 

2 2 

(ai|ai) = 2, (Q;i|a2) = -1, (a2|a3) = g, (a3la3) =-g- 

The highest root is 0 = 2 ai +302 + 30:3. The subalgebra g^ is the simple Lie algebra, isomorphic 

to G2, with simple roots {oi 02 + 03}. We choose the following basis of the coroot lattice L of 

a* • 

Uq 

71 = 01, 72 = 9 . 

We let / 3 i = 02, T = {/ 3 i}. One has: 


71 =71, 72 = 72 + /3i, p = --/3i +-72- 

Furthermore, A £ Pk,T iff, up to adding a multiple of < 5 , we have: 


eq5.20 


( 5 . 20 ) 


A = kAo + kiPi + A: 272 , where A:2 — ki, 2 k 2 £ Z. 


We consider the irreducible highest weight g-modules L{A) for the corresponding to H choice 
of positive affine roots. The following theorem is proved along the same lines as Theorem 15.41 


Th5.8 


Theorem 5 . 8 . The Q-module L{A) with the highest weight A such that (A|T) 
if and only if A £ (see h 5 . 20 \) ) satisfies 


0, is integrable 


k — 2 k 2 , 2 ki, 2 k 2 , ^2 — fci £ Z>o. 

I II 

Next, g' is the simple subalgebra of g^ , isomorphic to s^( 2 ), with the simple root 72. Hence 
p' = ^72, and conditions (i) and (ii) of Theorem 15.31 hold. It is also not difficult to check 
condition (iii). Hence Theorem 15.31 holds in this case as well. 


6 Remaining cases of modular invariance of modified normalized superchar¬ 
acters 

In Section 5 we found all cases of basic simple Lie superalgebras g (which are not Lie algebras), 
for which there exists a set of simple roots H and a maximal set of pairwise orthogonal (isotopic) 
roots T in H, such that the span of modified normalized super characters of integrable g-modules 
L(A) with (A|r) = 0 is 5L2(^)-ffivariant. It turned out that in the remaining cases, g = 
osp{ 2 m\ 2 n), m > n + 1 , and g = D{ 2 , l;a), there exist a set of affine simple roots H and two 
maximal subsets T and T' in H of pairwise orthogonal roots, such that the span of the union of 
modified normalized supercharacters of integrable g-modules L{A) with (A|T) = 0 or (A|T') = 0 
is 5L2(Z)-invariant. A similar situation occurs in the case g = osp{ 3 \ 2 ), subprincipal integrable 
g-modules. 
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6.1 Case g = osp (2m|2n), m > n + 2. 

The set of roots is described in terms of the same basis as is §5.4, and it is a subset of the set of 
roots of osp (2m + l|2n), obtained by removing the roots isj and Then L = Z{ej±ej | 1 < 
i < j < m} is the coroot lattice of = so{2m). Let uq G be the element, defined by 

O'o(£i) = -ei, O'o(en+i) = -En+I, andfJo(ei) = Si otherwise . 


Let T = {/ 3 j = I z = 1,... , n}, T' = (Jq{T) = {/ 3 ' = (7o{f3i) | i = 1,..., n}, and choose 

the following bases of L : 7^ = e* - for 1 < z < n, jn+i = £n +2 + En+i, 7i = £i - £i-i for 
n + 2 < i < m; and 7I = fJo(7i) for 1 < i < m. Then our basic condition (13.11) holds in both 
cases, the bilinear form (. | .) is normalized as in dH]), L = Leven,(T|r) = (T'lT') = 0, and 
condition (13.3p on k holds iff A: is a positive integer. 

We have: 


eq6. 1 (6.1) Pfc,T = ki[3i + kiSi I ki G -Z, ki — ki G Z, z = 2,..., m} + C6, 


2=1 


2=72+1 


and Pk,T' is obtained from replacing /3i by /3(. Hence we may consider the modification 
0A,r (resp. 0a,t') of a mock theta function 0A,r (resp 0a, t') for all A G Pfc,T (resp. Pk,T')^ 
where k G Z>o, and we have 5L2(Z)-invariance of the span of the set {0a,t | A G Pk,T} and 
the set {0A,r' I G Pk,T'}- Recall that 0a,t (resp. 0a, r') depends only on A mod (CT (resp. 
CT') +C5). Choose the following subset of simple roots: H = {ai = Em-i+i — ^m-i for 1 < 
Z ^ TTl n 1, Oim—n — ^n+1 tTm—n+1 — •••jClm+n—3 — ^2 ^2j O'm+n—2 — 

62 —Si, am+n -1 = Si—£i, oim+n = <5i+ei}, SO that 6 = £m+£m-i- Recall that h'^ = 2{m—n—l) 
and p = EI(Ln+2(* - ^ = <^o(p)- 

The following theorem is proved by the same method as Theorem 15.41 


th6.1 


Theorem 6.1. The'Q-module L{A) with the level k 0) highest-weight A, such that (A|r) = 0 
(resp. (A|T') = 0), is integrable if and only if A G Pk^r (resp. G Pk,T')! described by 116.1\) . 
satisfies 

k G Z^Q, ki G ki — ki G Z, 


kyyi P. ‘ ‘ ‘ P. k‘2 P_ II, k ^ k^y^ kyyi — k'2 — ki ki — k^, — 0. 


□ 


Denote by (resp. the set of all A G Pk,T (resp. G Pk,T'), such that the g-module 
L(A) is integrable and (A|T) = 0 (resp. (A|r') = 0). Note that Qk C consists of weights, for 
which A:i = 0. 

Note that g' is the subalgebra so(2m — 2n), corresponding to e^+i,... ,£m, and conditions 
(i) and (ii) of Theorem 15.31 hold. However, condition (hi) fails. For this condition to hold it 
suffices to consider the union of the sets Pk,T and Pk,T', and modify the proof of Theorem 15.31 
as follows. 

Since the modification of of chA depends on the choice of T, we shall denote the modified 
supercharacters by chA,T (resp. chA,r') if A G (resp. G D{,). The functions chA,T (resp. 
chA,T') depend only on A G mod (CT + C(5) (resp. A G mod (CT' + C(5)). 
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We have, in the notation of the proof of Theorem 15.31 formula (15.51) for A £ and the 


analogous formula for A G ni: 




eq6.2 (6.2) 


C^A,T' = (.9i) 9i (^0 ^ £+(«;)© 

i£l wGW' 


w{A+p),t/^ > 

IR-. 


L 

w(A+'p),T' 


Let = {A £ f^fcOf the form (16. ip with/cr.+i > 0} . Then, using (15.51) and (16.21) . by the 
same argument, as in the proof of Theorem 15.31 we obtain for A £ 


/iGQ/j. w^W^' 
mod 

cr+c <5 


o' uiew'- 
P&vt). 
mod 
CT'+C(5 


and a similar formula for A £ replacing in T by T' and VLk by Letting ch;^ = ch;^^^ 
(resp. ch;^ = ch;^^/) if A £ (resp. fl;.), we obtain from these formulas the following unified 

O 

formula for A £ flfc U 12^ : 


eq6.3 (6.3) ch)^{T, z,t)\^ = /{k + h'^)L'\ 


E 

/iGfifc mod (CTH-C(5) 

o ^ 

UQfc mod (CT'+C5) 


where 


wew'- 

and similarly for S(^x,T'),{p,T)j ^fc., replacing A + p by (To(A + p), etc. 

Denote by Dj, C 1}'* the set of all dominant integral weights for g\ Note that for A £ 
such that kn+i = 0 in its form (16.ip . we have A £ and ch^-p = ^AT'- Hence we may 

O ^ O ^ 

consider only A £ U D;-. For A £ (resp. D^) of the form (j6.ip we let 

m m 

a' = /cAo + hsi (resp. o-o(A') = /cAq - kn+iSn+i + hsi). 

2=n+l 2=n+2 

o ^ 

Then we have a bijective map ^ U —)• A i—)• z^a, defined by z^a = if G (resp. 

O ^ O ^ 

= (7o(A') if a £ Furthermore, for A,;U £ U we have 

S\,p = where h('^2+P ))^ z^i, z/2 £ D).. 

wew'- 

It follows from (j6.3p that Theorem 15.31 holds in this case as well if we replace mod (CT + C(5) 

O ^ 

in this theorem by the union of mod (CT + Cd) and mod (CT' + Cd). 
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6.2 Case 0 = osp (2n + 2|2n). 

II 

The set of roots, L,q^ and T = {/ 3 i, ...,/ 3 n} are the same as in §6.1 for m = n + 1. We choose 

the following basis of L : 7* = £„+! — for i = 1 ,... , n, 7n+i = 2e„+i. 

Choose the following set of simple roots 11 = {ao = (5 — (ei + (5i), ai = ei — 5i, a 2 = 

( 5 l — 82, 03 = £2 — 62, ■■■ , Oi2n-2 = ^n-1 “ Ot2n-l = — ^n, Ct2n = — ^n+l, « 2 n+l = 

+ En+i}- Note that p = p = 0. Let ao be the involution of the Lie superalgebra g, such 
that ao permutes ao with ai,a 2 n with a 2 n+ij and fixes all other roots from IT. Note that 
cro(ei) = 6 - ei,cJo(e„+i) = -£n+i,ao{8i) = e* for i / l,n + 1, ao{dj) = 6j. We normalize Ai 
by the condition |Aip = 0, so that Ai = Aq + ei — and do permutes Aq and Ai. We let 
T' = ao{T), f3^ = ao{/3i), 7 ' = 0 - 0 ( 7 *). The following theorem is proved by the same method as 
Theorem 15.41 


Th6.2 


Theorem 6.2. The Q-module L(A) with the level k (^Q) highest weight A, such that (A|r) = 0 
(resp. (A|r') = Oj, is integrable if and only if, up to adding a multiple of 6, 


n n 

A = kAo + kn+i£n+i + ^ kil3i {resp. = kAi - kn+i8n+i + ^ 

i=l i=\ 


satisfies 

k G ^>0) k n+l £ 2 ^’ ^n+l £ 

ki > k 2 > ■ ■ ■ > kn > |A;n+i|, k > ki + k 2 , and k = ki + k 2 ^ ki = k 2 . 


□ 


Denote by (resp. Q!jf) the set of level k weights A, orthogonal to T (resp. T'), for 
which the g-module L(A) is integrable. In order to write down the formulas for the modified 
normalized supercharacters for A E (resp. A' G D'^), introduce the following notation: 


p p n 

Cp — ^ ^ 8j , dp — ^ ^ dj, ^ (t, 8p, £*1+1 T ^p—i dp'). 

j=i j=i p=i 

Then, by formula (j5.3p . we have, in notation of Theorem 16.21 


eq6.4 


(6.4) 


R Ch^ = ^ e (u;)w^ (02fc„+i,2fc(T, Cn+l - dn)^fc) , 


eq6.5 (6.5) 


R ch^/ = e 


2'Kikt 


E 

uGW* 


{w)w (02fe„+i+2fc,2fc(Den+l -dn)^k 


Since the span of the theta functions 

{02fc„+i,2fc) 02fc„+i+2fc,2fc I kn+1 G 1^’’ ^ — “^^ri+l E fc} 

is S'L 2 (Z)-invariant and ‘hfc is S'L 2 (Z)-invariant, we conclude that the span of all modified 
normalized supercharacters ch^ for A G Dfc and ch^/ for A' G is S'L 2 (Z)-invariant. 
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In order to write down the explicit transformation formula, note that 


7 Ic Jc Ic 

Qk n = {^^0 =t 2 ^^+1 2 ^ ~ ^ 2^”"'"^ 2 ^ 

z=l i=l 

o ^ 

and let = ri'^/(flfc n For A = /cAq + fen+i^n+i + Yl'i=i ^ (resp. A = /cAi - 

O ^ 

^n+lEn+l + XlILl ^ ^fc) 


Ux = kn+iEn+i (resp. = (/C + kn+l)en+l)- 


O ^ 

Then for A G U (disjoint union) we deduce, from the transformation formula for theta 
functions (see e.g. (A5) from [KW6j l. equations (j5.6p and Theorem 11.31 (a) the following 
transformation formulae: 


eq6.6 


( 6 . 6 ) 


1 z \z\ 

cliA — 

T T It 


1 

2x/k 


mod CT+C(5 

o ' 

U Q}. mod CT'+C(5 


ch;^ (r + l,z,t) = e fc i^clix {T,z,t). 


6.3 Case g = D{2, 1; o), a G Q, —1 < o < 0. 

Recall that the Lie superalgebra D{2, 1; a) is a family of simple 17-dimensional Lie superalgebras 
with the even part isomorphic to s£( 2 ) 0 s£( 2 ) 0 s£( 2 ), depending on a parameter a / 0, —1. 
Two members of this family are isomorphic iff the corresponding values of parameters lie on 
the orbit of the group, generated by transformations a —1 — a, o. ^ |Klj . Hence, provided 
that a G Q, we may assume that —1 < a < 0, and write it uniquely in the form: 


eq6.7 


(6.7) 


a = -, where p,q € Z>o are coprime. 

p + q 


We consider the set of odd simple roots H = {ai, 02 , 0 ( 3 } of g, with the following non-zero 
scalar products: 

{ai\a 2 ) = a, (aijas) =-(a 0 1 ), (0:2103) = 1 - 

The highest root is 0 = oi 0 02 + 03 , and {6\9) = 0; we also have p = 0, /i'^ = 0. We let g^ be 
the subalgebra of gg, isomorphic to s£{2) 0 s£{2) with the simple roots oi 0 02,01 0 03 . The 
corresponding coroot lattice is L = 0 Note that, due to (16.7p . this lattice is 

negative definite. 

In this section we consider non-critical (i.e. of non-zero level) irreducible highest weight 
g-modules L(A) for the corresponding to H choice of positive affine roots. Such a module is 

called integrable if it is integrable with respect to g^ . The following theorem is proved along 
the same lines as Theorem El 


Th6.3 


Theorem 6.3. Let A = level k = Then 

(a) The Q-module L(A) is integrable if and only if the following conditions hold: 
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mi+m 2 ^ mo +1713 ^ _!M±p 2 non-uegative integers; 

(a) mi + ruj = 0 =► mj = nij = 0 for i = 0 or 1, j = 2 or 3. 

(b) The g-module L{A) is ao + ai fresp, + a^)-integrahle iff 


(i) mo + mi (resp. m^ + rnz) S Z>o ; 

(ii) rriQ + mi (resp. m 2 + m 3 ) = 0 ^ mg = mi = 0 (resp. m 2 = m 3 = Oj. 


Core. 4 


Corollary 6.4. If there exists an integrable D{2,1; a)-module L{A), such that its level is non¬ 
zero, then a G Q, and we may assume that — 1 < a < 0. so that a is of the form In this 

case 


eq6.8 


( 6 . 8 ) 


k 


Pqri u- • ^ 

- for some positive integer n. 

p + q 


□ 


From now on we shall assume that a is of the form (|6.7I) and k is of the form (16.8p . We shall 
study the integrable level k 0 -modules L{A), such that (A|/3) = 0 (resp. (A|/3') = 0), where 
/3 = oi,/?' = oq. (These conditions mean that mi = 0 (resp. mg = 0)). 

Let T = {/3} (resp. T' = and choose the following Z-bases 71,72 (resp. 7 ^, 72 ) of the 

lattice L, such that ()3.1h and (I3.2h hold: 


71 


oi + 03 
a -\-1 


, 01 - 1-02 
= -7i, 72 =- 




Ol -I- 03 
a 1 


/ 

= 72- 


Then we have 71 = 71 = - 7 ' and 72 = -^{{p + q)ai + qoi 2 - pas), 72 = -^{{p + q)ai + 

{2p + q)ai Apas). 

The Lie superalgebra g has an involution erg, which permutes og with oi and 02 with 03. 
We choose Ai = cro(Ao). Note that crg(/ 3 ) = ff . 

Denote by (resp. the set of all level k weights A, such that the g-module L(A) 
is integrable and (A|/?) = 0 (resp. (A|/3') = 0)). We deduce from Theorem 16.31 the following 
corollaries. 


Cor6.5 


Corollary 6.5. (a) Let A G Then, up to adding a multiple of 6 , we have 

pq 


eq6.9 (6.9) 


A = A:Ag — 


^272 + kiai, 


2{p A qY 

where ki,k 2 GT, satisfy one of the following conditions: 


(i) —pn < /c 2 < qn, max{ 0 , —/C 2 } < ki < min{gn — k 2 ,pn}; 

(ii) ki = 0 , k 2 = qn; 

(Hi) ki =pn,k 2 = —pn. 

(b) Let A G LlY Then up to adding a multiple of 6 , we have 


eq6.10 


( 6 . 10 ) 


A = A:(Ao -h 7i) - 2(p_gg)2 ^272 “ 


where /ci, A :2 G Z satisfy one of the following conditions: 
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Core. 6 


Core. 7 


Rome.8 


Conje.9 


eqe.ll 


eqe.12 


(i) pn < k 2 < pn + {p + q)n, max{(/n, /c 2 — pn} < ki < min{(p + q)n, k 2 — pn + qn}] 

(a) ki = {p + q)n, k 2 = {2p + q)n] 

(Hi) ki = qn, k 2 = pn. 

□ 

Corollary 6.6. Let A G be of the form \6.9\) . Then 

(a) L{A) is oq + ai-integrable iff one of the following three conditions holds: 

(i) k 2 = qn — {p + q)s, where s G Z, 0 < s < n, and ps < ki < min{(p + q)s,pn}] 

(a) or (Hi) from Corollarv \6.5^ a). 

(b) If —pn <k 2 < qn, then there exists ki, such that L{A) is not oq + ai-integrable. 

(c) L{A) is 02 + a^-integrable iff A = /cAq (i.e. ki = k 2 = 0). 

□ 


Corollary 6.7. Let A G be of the form id.lOi) . Then 

(a) L{A) is oo + ai-integrable iff one of the following three conditions holds: 

(i) k 2 = s{p + q) — qn, where s ^ Z, n < s < 2n, and ps + {q — p)n < ki 
< min{(p + q)n, s{p + q) — pn}; 

(a) or (Hi) from Corolla, rv \6.5\ (b). 

(b) If pn < k 2 < (2p + q)n, then there exists ki, such that A is not oq + ai-integrable. 

(c) L{A) is 02 + a^-integrable iff A = kAi (i.e. ki = qn,k 2 = {p + q)n). 

□ 


Remark 6.8. It follows from Corollaries 16.61 and [Q that A (A) with A G U cannot be 
simultaneously oq + ai-integrable and 02 + os-integrable. 


Conjecture 6.9. Let L{A) be an integrable level k (f^ 0) Q-module, such that A G (resp. 
AgQ'J. LetW = W*{= (^ai+ 02 ) ^ai+ 03 )) L{A) is neither uq + ai-integrable, nora 2 + 0 ( 3 - 
integrable, and let W = {W'^,ra) if L{A) is either a = oq + ai-integrable, or a = 02 + 03- 
integrable. Let ca = 2 if A G Llk (resp. A G and in cases (ii) or (Hi) of Corolla,ru \6.5\f a) 
fresv \6.3\f b) ). and let ca = 1 otherwise. Then the normalized supercharacter of L{A) is given 


by the following formula, similar to 14.12^: 


( 6 . 11 ) 


R- 


r/, ^ ('“^)^(®A,T(resp. T'))- 


CA 

wGW 


As before, the formula for the modified normalized supercharacter ch^ is obtained if we 
replace mock theta function 0^ by its modification 0^ in the RHS of (j6.1ip . By ()3.7p . we 
obtain the following explicit formulas for the modified mock theta function: 

( 6 . 12 ) 0 A,T = e^’"*^*0fc2,(p+g)n('r, + (a + 1)02 + aas) -«!, -03), 
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eq6.13 


Prop6.10 


(6.13) -ai + (a - l)a 2 + aas) - 02 ). 

Using these formula, we can write down explicit formulas for the modified normalized super¬ 
characters. 

Proposition 6.10. Let A € 11^ be of the form 116.91) . and let N = {p + q)n. 

(a) If A (A) is neither oq + ai-integrahle nor 02 + a^-integrable, then 


R ch^ = ^0fc2,Ar(u ai + (a + l)a2 + aas) -oi,-03) 


+Qk 2 ,NiT, -ai + {a - 1)02 + aas) -02)) • 

(b) If L{A) is aQ + ai-integrable, then 

R~(Rj^ = ({&k2,N + &-k2+2qn,N){T,ai (o + 1)02 + aas)^!^”'!( t, -oi, -03) 

CA V 


+ (0fc2, N + 0-fc2-i-2gn, nX '^) “0^1 + (« — l)a2 + (r, 6, — 02 )^ • 

(c) In cases (ii) and (in) of Corollarv \6.5\ (a) we have: 

R-(Qk2,NiT,ai + (a + 1)02 + 003) -oi,-03) 

+ 0 fc 2 ,Ar('r, -«! + (a - 1)02 + aas) - 02 )) • 

(d) If L{A) is a 2 + oi^-integrable, i.e. A = kAo (by Corolla, rv \6. 6\f c)). then 

.R“ch^ = e^’"**^* (^ 0 o,Ar(T, oi + (a + 1)02 + 003) ““s) 

+ 0 o,Ar(T, —Oi -I- (a - 1)02 + 00 : 3 ) 0,-02)^ • 


Core.11 


□ 

O 

Let llfc = {A S flfcl either Ais not (ao+«i)-integrable, or(A;i,/c2) = (0,gn) or = (pn,—pn)}. 

O _ 

Corollary 6.11. If A £ is of the form li6.9\) . then the range of k 2 in A is {j G Z| — pn < 
j < <?ra}, and we have: 

R-chj^ = (^0fc2, (p+q)niT, Oi + (a + 1)02 + 003) -ai, -«3) 

+0fc2, {p+q)nir, -ai + (a - 1)02 + 003) (r, 9, -02)) • 

□ 


Prop6.12 Proposition 6.12. Let A G be of the form < 1 6. 1111) . and let N = {p + q)n. 
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(a) If L{A) is neither oq + ai-integrable nor 02 + a^-integrable, then 


R ch^ = (^ 0 _fc 2 ,Af('r, ai + (a + 1)02 + 003) -ai,-as) 

+0_fc2, Ar(r, -ai + (o - l)a2 + aas) (r, 9, - 02 )^ ■ 

(b) If L{A) is ao + ai-integrable, then 

R-ch^ = ((0_fe2,iv + Qk 2 + 2 qn,N){r,ai + (0 + l)a2 + aas) -ai,-as) 

CA V 

+(0_fc2,Ar + &k 2 + 2 qn,N)ir,-ai + (a - l)a2 + a3) 0,-a2)^ . 

(c) In cases (ii) and (in) of Corollary \6.5\f b) we have: 

^“ch^ = ^0_fc2,Af(T, ai + (a + l)a2 + aas) -ai, -as) 

+0_fc2,Ar(T,-ai + (o- l)a2 + aas) 0,-a2)) . 

(d) If L{A) is a 2 A a^-integrable, i.e. A = kAi (by Corollary \6.T\ (c)), then 

(^0Af,Ar(T, ai + (a + l)a2 + oa3) -ai,-as) 

+C)n,n{t, -ai + (a - l)a2 + oas) (r, 0, -a2)^ • 


□ 


Core.13 


Let = {A £ A is not (ao + ai)-integrable}. 

Corollary 6.13. If A G is of the form i6.10\) . then the range of — ^2 in A is {j G Z 
{2p + q)n < j < —pn}, and we have: 


R ch^ = (^0_fc2,(p+g)n(T,ai + (a + l)a 2 + aas) -«!, -as) 


+0_fe2, (p+q)n('r, -Oi + (a - l)a2 + as) (r, 0 , -02)) . 

□ 

00^ 00^ 

Note that n = 0, and define the map 1 / : U ^ Z, letting z^(A) = k 2 (resp. — ^ 2 ) 

O O ^ 

if A G Ofc (resp. 11^)- Then 

O O ^ 

{i/(A)| A G rifc U = {j G Z I gn — 2{p + q)n < j < qn} . 


Hence we obtain the following theorem. 
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Th6.14 


^ _ o o ^ 

Theorem 6.14. The span of the modified normalized characters ch;^ with X G U is 
SL 2 {'Tj)-invariant, and the transformation formulae are as follows: 


1 |zr, 

chA(--,-,i- 17-) 
r T It 


1 


e (r, z, t), 


~ _ TTZ ~ — 

ch;^ (r + 1, z, t) = e2"(p+9) (r, t). 


□ 


6.4 Case g = osp(3|2), subprincipal integrable g-modules 

Here g is the same as in Section 5.3 with m = n = 1. As there, the set of roots is described 
in terms of an orthogonal basis ei, (5i of f), such that (eijei) = —(5i|(5i) = We have Ag = 
{±2ei, ±(5i}, Aj = {±ei±5i, ±ei}. We choose the following set of odd simple roots of g and 
of g : 

n = {ai = 5i — El, 02 = £i}, n = {oq = d — El — (5i, oi, 02 }- 

1 1 11 

The highest root of g is 0 = ei + <5, and p = —^oi, p = 5(^0 — «i)- We let g^ be the subalgebra 
of gg, isomorphic to s(.{2) with the simple root oi + 02 (which is different from that in Section 
5.3 for m = n = 1). The corresponding coroot lattice L = Z7, where 7 = 4(ai + 02)5 is negative 
definite. We let /? = oi, T = {fi}. 

In this section we consider non-critical (i.e. of level k —^) irreducible highest weight 
g-modules T(A) for the corresponding to H choice of positive affine roots. Such a g-module is 
called subprincipal integrable if it is integrable with respect to gg . This means that T(A) is 
OiQ + a:2-integrable and oi -|- a2-integrable. 

Let (To be the involution of the Lie superalgebra g, such that (7{ao) = ai and o-(a 2 ) = a2- 
We normalize Ai by the condition |Aip = 0, so that ctq^Aq) = Ai = Ao-|-q;o~ch and (To(p) = p. 
Let 7' = (To(7) = ao + 02, fd' = (To(/ 3) = ao, T' = {fi'}. Note that ao maps g^ to itself (though 

II 

doesn’t keep g^ invariant). 

In this section we consider irreducible highest weight g-modules L(A) for the corresponding 
to n set of positive affine roots. We shall also assume, as before, that the level A: of A is not 
0. Denote by (resp. Xl'jf) the set of highest weight A of subprincipal integrable g-modules 
of level k such that (A|T) = 0 (resp. (A|r') = 0). The following theorem is proved along the 
same lines as Theorem Eai 


Th6.15 


Theorem 6.15. Let A G (resp. Then 


(a) Up to adding a scalar multiple of 6, A is of the form 


eq6.14 


(6.14) 


A = kAo +—ai{resp. = fcAi-|--^ oq), 


where the following conditions hold: 

(i) k G |Z, k < —1; 

(a) m G Z, 0 < m < —{4:k + 2), or k = —m = 1. 
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(b) Subprincipal integrable'Q-module L(A) of the form {6. If ) is 2a2-integrable iffm = 0, i.e. 
A = /cAo (resp. kki). 

(c) Subprincipal integrable Q-module L(A) with A of the form (6.If) is 6 — 2a2{= cto + ®i)' 

integrable iff k ^ ^^<05 < m < —Ak — 2 and m = —2k mod 2 Z. 


□ 


Core.16 


Corollary 6.16. (a) If k £ |Z<_ 2 , and A = kAo + (resp. = kAi + y^^o), where m = 

0 or 1, then A E (resp. H'f,) and A is not 5 — 2a2- integrable. 


(b) If 2k ^ Z, then (resp. Vl'^) n{5 — 202 — integrable A} = 0 . 


□ 


Note that for A £ Hk (resp. H'ff) we have: 


0 




pA+p 


A+p,T (resp. T') ‘'7"^ _ (resp. -ao) 

"fGL 


and let 

^A+p^,T(resp. T') = 0 A+p,r (resp. T') “ ^ai+02 0 A+p^,T (resp. T')- 

The character formulas for subprincipal integrable g-modules, proved in [GKj . can be rewritten 
as follows. 


Prop6.17 


Proposition 6.17. Let A £ (resp. 


(a) If A is neither 2a2-integrable, nor 6 — 2a2-integrable, then 


R ch^ — {resp. T')- 


(b) If A is 2a2-integrable, then 

R ch^ = I {resp. T') + ^a2^A+p,T {resp.T')] ■ 

(c) If A is 6 — 2a2-integrable, then 

R ch^ = jn/) — rao+alA/^_^_■pY {resp.T')- 


□ 


Next, we introduce the following coordinates on t) : 

h = 27 ri(-TAo + ^ 1(01 + 202) + 2201 + t5) = (r, 21 , 22 , t). 

Then the functions (j-egp_ j’') where A is of the form (I 6 . 14 p . can be expressed in terms 

of the functions 


eq6.15 


( 6 . 15 ) d/[^’*l(r,2i,22,t) ($+['^'*l(r, 21,22) - 4 >+[^;^l(r,-22,-21)) , 
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where cj)+[^;^] ig defined by (| 1 . 10 p . as follows: 


eq6.16 (6.16) 


^A+p,T = ^' 2"'1(2 t,Zi,Z2,-), 


eq6.17 (6.17) 


Ak+p,T' = e-"( 2 fc+i)(zi+z 2 +T) ^[-{2k+i)-,Kp]i^2T, Zi + r, 2:2 + r, -). 


Recall that in Section 1 we defined a modification of the function (^+[^i®l(r, zi, 22 ), denoted 
by (l)+[^’^l(r, 2 : 1 , 22 )) for any M £ iZ>o,s £ ^Z. Denote by the function, obtained from 

^[M;d, replacing by in (|6.15l) . 

Due to Theorem 16.151 if A £ Dj, U and k ^ (i.e. k is non-critical), we have 

— {2k + 1) £ ^Z>o and £ ^Z in (j6.16p . (16.171) . Hence we can define modifications 

^A+p^,r (resp. r')(D ^i) ^ 2 ) 0) replacing 'h by 'h in (j6.16l) and (I6.17p . Hence, by Proposition 

16.171 we can define the modified normalized supercharacters ch^, replacing A by A in this 
proposition. By Corollary 11.61 fa) the function £ ^Z>o,s £ ^Z, depends only on s 

mod Z. Thus we obtain the following corollary. 


Core.18 


Corollary 6.18. The modified normalized supercharacter ch^ for A £ U k 7 ^ — of 
the form depends only on k, and on m mod Z. Consequently, the span of all modified 

normalized supercharacters ch^ for A £ (resp. k 7 ^ — is spanned by ch^^^j and 

chfcAo+iai ("^esp. &2 /chfcAj and □ 


LemmaG.19 


Lemma 6.19. For M £ ^Z>o,s £ ^Z one has: 


2^-[^;-l(2r,2i,22,t) 


^[2M;2s]^^^ £1 ^ £2 ^ g-27ri^^[2M;2d(.^^ 


2l + 1 


22-1 t 
2 ’2 


□ 


By Proposition 16.171 formulae (I6.16p . (I6.17p . and Lemma [6. 191 we obtain for k £ |Z, 
k<-\: 


eq6.18 

(6.18) 

clifcAo = 

lifl 

- / 2 ), 


= ^(/l + / 2 ), 




1 (A 



= -\{h + h) 

eq6.19 

(6.19) 

chfcAi = “ 

-h), 

^^fcAi + ioi 


where (we can remove the second superscript in T due to Corollary II.4p 


{R~ fi){r, zi,Z2,t) 
(R"/ 2 )(r, 2 l, 22 ,t) 

{R~f3){r, Zi,Z2,t) 
{R~ ffi){T, Zl,Z2,t) 


= $[-4^-2]/ A ^ C 

^ ’ 2 ’ 2 ’ 4'^’ 

= ^[-4fc-2]/ ^1 + 1 ^2 + 1 t 


AH 


’ 4 ^’ 


_ g-7ri(2fc+l)(2:i+22+T) 4,[-4fc-2] 2l + T 22 + T t 

_ ^-TTi{ 2 k+l){zi+Z 2 +T) ^[-4fc-2]^^ 


2 2 4 

21 +T + l 22 +r + l 

9 ’ 9 ’4^ 
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From Theorem ll.3f a) and (jS.hD we obtain: 


eq6.20 


( 6 . 20 ) 


hi- 

1 

5 

2:1 

22 

, t + 

^ 122 . 

= rfi{T, 

2 : 1 , 

• Z2,t), 

r 

T 

T 


T 


hi- 

1 

5 

2:1 

Z2 

, t + 

Z 1 Z 2 . 

= rfsir, 

Zi, 

<Z2,t), 

T 

r 

T 


T 


hi- 

1 

? 

2:1 

5 

22 

, t + 

ZlZ2-^ 

= rhir, 

Zi- 

• Z2,t), 

r 

r 

r 


T 


hi- 

1 

? 

T 

2:1 

5 

r 

5^2 

r ' 

, t + 

2i22^ 

r 

= (-l)^'''r/4(r,2i,22,t) 


eq6.21 


fi{T + l,Zl,Z 2 ,t) = fi{T,Zl,Z 2 ,t) for i = 1,2, 
(6.21) f3{T + l,Zi,Z2,t) = -{-i)^^f4iT,Zi,Z2,t), 

Uir + l,Zi,Z2,t) = -{-i)^’^f3{T,Zi,Z2,t). 

Corollary 16.181 along with (I6.18p - (I6.21I) . imply the following theorem. 


Th6.20 


Rm6.21 


Theorem 6.20. The span of the modified normalized supercharacters ch^ for A G U 
where k G k < —is a 4-dimensional SL 2 {T>)-invariant space, spanned by /i, / 2 , /s, f^. □ 

Remark 6.21. The maximal non-critical k, for which U is non-empty, is k = — Note 
that for this value of k, we have —Ak — 2 = 1, and that ^i, 22 ) = ‘^Ldd (f ~Z 2 , —^i) (see 

(jl.l3p l. hence Furthermore, by the denominator identity for s.^(2|l) we have: 


TW(r, 2 : 1 , 22 , 0 ) 


.r]{Tfdii{T, zi + Z2) 

2 ,_ 

'i 9 ii(r, 2 i)i?ii(t, 22) ’ 


Hence, using the elliptic transformations of the functions 'dab (see e.g. Appendix of |KW6| ) we 
obtain: 


{R /l)(r, 2 l, 22 ,t) 


[R / 2 )(r, 2 i, 22 ,t) 


{R h){r,zi,Z2,t) 


{R /4)(r,2i,22,t) 


— IP 2 -=- 

^9ll(T,f)l?ll(T,f)’ 

^ i9io(r,a)i?io(r,f)’ 

. _zLrt ??(r)^i?ii(r, 

' i?oi(T,a)^?oi(T,f)’ 

. r?(r)3i9oo(r,a±£2) 

— IP 2 -=- 

^00(T,f)^?00 (T,f)- 


Since for osp (3|2) we have by (14.Sp : 

i?“(r, 21 , 22 , t) = -ie 


??ll('r, 2 i)i?ii(t, 22)'i?ll(T, ^J-^) ’ 


all four functions fj, up to a factor ±ie 2 ^ are ratios of products of the theta functions dab- Of 
course, these four functions satisfy the transformation properties (16.201) . (I6.2ip with Ak replaced 
by -3. 
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6.5 Fermionic construction of osp (M|N)-modules 

Let V = Vq®Vi he a, vector subspace with dim Vq = N = 2n even and dim Vi = M, and let 
(.|.) be a non-degenerate super-skewsymmetric bilinear form on V (i.e. it is skewsymmetric 
on Vg, symmetric on Vj and (VglFj) = 0), and let g = osp {M\N) be the corresponding ortho- 
symplectic Lie superalgebra. Let be the corresponding to (F, (.|.)) vertex algebra of free 
superfermions. 

In our paper |KW3| . Section 7, we constructed a representation of g in the superspace 
which is integrable with respect to the Lie subalgebra so (M). We showed that (if (M, N) ^ (1, 0) 
or ( 2 , 0 )), this g-module decomposes in a direct sum of two irreducible level 1 highest weight 
modules. Note that these g-modules are integrable if M > N -|- 2 and subprincipal integrable 
otherwise. As we pointed out in |KW3| . Remark 7.3, the normalized supercharacters of these 
g^-modules are theta functions, which, however, do not span an S'L 2 (^)-invariant space. In 
this section, using the twisted superfermions, we shall construct the missing, level 1 g-modules, 
integrable with respect to the affine Lie subalgebra so(M), which restore the S'L 2 (^)-iiivariance. 
For information on twisted helds we refer to e.g. [EM]. 

First, consider the case M = 2m -|- 1 is odd. Choose a basis R = Rg U Rj of V, where 
Rg = {(^®, = 1,..., n} is a basis of Vg and Rj = {V’, V'*; = 1) ■ ■ ■ > is a basis of Fj, 

snch that all non-zero inner prodncts between basis elements are: 

= 1, = mn = h im = i- 

Let denote the vector superspace with an even vector |0) and operators for each a G R 
and n G Z, such that 

(i) [ar,bs] = {a\b) 5r-sl, a,b e B, r,s eZ, 

(ii) ip\.\h) = V’rlO) = 0 for r > 0 , ip'‘*\h) = '0^10) = V’r|0) = 0 for r > 0 , 


(iii) is irreducible with respect to all the operators an, a G B, n £ Z. 


The helds 


«(^) = E 




ar^z 






are called twisted superfermions', dehne the annihilation 


part a{z)- = En-a^|o>=o^ i'b® creation part 0 ( 2 :)+ = a(z) — a(z)-. For two such 
helds a{z) and b{z) their normally ordered prodnct : a(z)b(z) := Erez • -r is computed 

by the following formula (see |KW5] . (1.14)): 


; ab :r= ^ {as,-k-lbr-sa+k+l + i-lT^''^^^’’^br-sa-kasa+k) 


Sa(Q'|fc)(5f^0) 


where Sa = —^ (resp. |) for a = and (/?* (resp. for a = ' 0 **, (/?**, and ip). 

We constrnct a representation of the Lie super algebra g in the snperspace <I>^ as follows. 
Given b G g(= osp (2m-|-l|2n)), denote by b{z) = i'b® corresponding current in 

g. For the choice of simple roots of g as in |KW3| . Table 6 . 1 , let Cj, hi, fi, i = 1,2,... ,m + n, he 
the Chevalley generators of g, and let eo G g_0, /o G Qe be such that [eo, /o] = —ho, \ho, eo] = 2eo 
[9 is the highest root of g). It is straightforward to check that the following formulas dehne a 
representation of g in the snperspace with K = 1 and d, such that d|0) = 0, [d, Un] = —nOn 
for a G R, n G Z: 

eo{z) = {z)lp^* {z)', fo{z) = -^ip^{z)ip^{z)', ei{z) = (p^(z)(p"~^^* (z) for i = 1 ,..., n - 1 ; 
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en{z) = en+iiz) =i = 1,... ,m - 1; en+miz) = ip'^{z)'ilj{z); 

ho{z) = - : {z)(p^* (z) :; hi{z) =: (p\z)ip"* {z) : - : : fori = 1 ,... ,n - 1 ; 

hn{z) =: ip'^{z)ip'^*{z) : + : {z)'il;^* (z) :; hn+iiz) =: 'il^\z)'ij;"* {z) : - : : 

fori = 1 ,... ,n - 1 ; K+miz) = 2 : {z)'tp'^* {z) : . 

Obviously, we have a g-module decomposition = $even ®^odd> where (resp. 

is the span of monomials in a^, o € i?,n S Z, of even (resp. odd) degree. Moreover, using the 
same method as that in the proof of Theorem 7.1 of |KW3] . one can show that the g^-modules 
‘hgygn and are irreducible, with highest weight vectors (for the choice of simple roots as 
in [KW3] i |0) and V'olO) respectively. It is not difficult to show that with respect to the choice 
of simple roots and the normalization of (. | .) as in Section 5.4, both modules have highest 
weight —Am+m and that for this choice of simple roots, we have the following isomorphisms of 
g-modules: $even — L{Aq), 4>odd — L{Ai). By Theorem 15.61 provided that m > n + 1, the three 
modules L(Ao),T(Ai), and L{—Am+n) are all so{2m + l)-integrable g-modules of level 1, such 
that the highest weight vanishes on T. 

Recall that in the case M < N + 2, the bilinear form, considered in Section 5.4, is multiplied 
by —see Section 5.2. In particular for g = osp(3|2), considered in Section 6.4, we obtain 
the (critical) level k = —^ g-modules © -^(“^-^i)) and = L[A 2 ) © L{A 2 ). 

Due to Theorem 16.151 the three g-modules L(—^Aq), T(—^Ai), and L{A 2 ) are all subprincipal 
integrable g-modules of level — such that the highest weight vanishes on an odd simple root. 

In [KW3| we computed the character of the g-module 4>y. In the same way we compute the 
supercharacter of , hence of L{—Am+n), in notation of Section 5.4: 


ch 


= ch 


odd 


= e 




n 

k=l 


nr=i(l “ 


Using this formula, we can write down in the coordinates 


eq6.22 


( 6 . 22 ) 


m n 

h = 27ri(-TAo - ^ XiSi + ^ mSi + t6) 
i=l i=l 


the following formulas for the the normalized supercharacters: 


eq6.23 


(6.23) 


eq6.24 


(6.24) 


eq6.25 


(6.25) 


(chAo + chAj(r, x,y,t) = 

(ciiAo - cliAi)(r, x,y,t) = 
ch:A^^Jr,x,y,t) = (-z)V-* 


7?(r)^r?(r)^ ’ 

?/(i)^( 2 r) 


r?(r) 

y{2T 




virY 

-r/(r) 


ni^i^oo(u Xj) 

nlLi^oo(r, 2 /.)’ 

nr=i^oi(T,y.)’ 

n^i^io(u Xj) 
niLi^?io(uyi)' 


Consequently, from the known modular transformation formulas for /^(r) and 'dabix,z), see 
e.g. Appendix of [KW6] . we obtain, by (I6.23p - (l6.25p . modular transformation formulas for the 
normalized supercharacters: 


(chAo + chAjIg = chAo + chA^, 
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(cl^Ao - cV)ls = 

= ^(-*)"(chAo - C^Ai), 

and the three normalized supercharacters are eigenvectors with respect to the transformation 
{T,x,y,t) !-)• (r + l,x,y,t) with the eigenvalues and 

respectively. 

Thus, the span of these three normalized supercharacters is 5L2(Z)-invariant. Likewise, the 
span of the subprincipal integrable critical level k = —| osp (3|2)-modules L(—^Aq), L(—|Ai), 
and L(A 2 ) is 5L2(Z)-invariant, with the transformation matrices obtained from the above by 
letting m = n = 1. 

The case M = 2m is quite similar. The basis Bq of Vg is the same, and the basis of is 
Bi = {?/)*,'(/!**I i = 1,... ,m}, with the same inner products. The space is constructed in 
a similar way. The construction of the representation of g in , where g = osp(2m|2re), is 
constructed similarly. The only change is in the following formulas: 

en+m{z) = 

hn+m{z) = ■.^P^-\z)'iP^-^*iz): + :i;^{z)^^*iz): . 


Then the g-modules <heven are irreducible, and, for the choice of simple roots as 

in |KW3j . the highest weight vectors are |0) and with weights A^+n and A^+n-i 

respectively. Consequently, for the choice of simple roots as in Section 5.4, we obtain the 
following isomorphisms of g-modules: 

$even = T(Ao), $odd = ^(Al), ^even = H^m+n), ^odd = T(A^+„_i). 


By Theorem 16.11 for m > n we thus obtain all integrable g-modules of level 1 with highest 
weights vanishing on T or Th 

The supercharacter formula for <1>^ is similar to that in the case of odd M : 


ch 


± ch 


-^-odd 


= e 


n 

k=l 


nr=i(l T =F e^iq^)' 


Using this formula, we can write down the normalized super characters in coordinates (|6.22p as 
follows: 


eq6.26 


(6.26) 


eq6.27 


(6.27) 




— ch“ )(t r V f) — r^TTit (- \n-m nt=i ^01 (u ^i) 


eq6.28 (6.28) 


(chA„+^ +ch^„+_i)(ua;,yT) = i-iTe r?(r) 


r-m WT=l^w{T, Xj) 

nr=i^io(T,2/.)’ 
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eq6.29 (6.29) 


(ch. — ch. 


)(r,x,y,i) = (-l)^-e2-S7(r) 


Xi) 


nr=i^ii(T,y.)’ 

The modular transformations of these normalized supercharacters are as follows: 


i^Ko+^K,)\s 

i^Ko-^KJ\s 


chXo + chX^ 

(-i)^(chXg -chXj, 
(-^)""”(chA.+^ - chX„^_ J 


and the four normalized supercharacters are eigenvectors with respect to the transformation 
{T,x,y,t) I—)• {t + 1, x,y,t) with the eigenvalues 

gfj(n-m)^ _gf|(n-m)^ ^-^(n-ni) ^ ^-^(n-ni) 

respectively. 


6.6 Modular invariance of characters from that of snpercharacters 


We show here that the modular invariance of modified normalized super characters implies 
the modular invariance of modified normalized characters together with the modified normalized 
characters of super characters of twisted modules, i.e. of 0 *"'-modules, constructed in |KW 6 ] . 
Section 4. 

Recall that ch^^^^ is expressed via formula (14.41) in terms of ch^^^^, where ^ G 1) is a fixed 
element, such that q;(^) G Z for all a G A. Hence we have 


eq6.30 


(6.30) 


ch.'j^{h) = e ^^*^^^^ch^(/i + 27 ri^). 


Also, we have, by [KW 6 j . (4.7), the ^'^-modules, for which the normalized (super)characters 
are: 


eq6.31 


(6.31) 


= ch^(t_ 5 (/i)) 


eq6.32 


Since 7 i(^) G "E, I3i{^) G ^ + E, it follows from (I6.3nh and (I6.31jl the following equalities for 
modified characters in coordinates (HAD: 

elixir, z,t) = e"^’"*^(^)ch^(r,^; + ^,t), 

(6.32) ch^’ (r,z,t) = chA(r,z + rC,t + (?|2) + ^ICP), 

^T’^ix,z,t) = e"^^*^(^)ch^(r,2: + rC + ^,t+ (^|2:) + ^I^P)- 


It is straightforward to deduce from (j6.32D the modular invariance of the family 


{dil, ch^’"^, ch^’ }Aefi, 


provided that the family {ch^jAeo is modular invariant. 
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Prop6.22 Proposition 6.22. Suppose that, for X ^ Q. we have 


= ^ *S'A;,ch^ and ch^ 

^ iM&n 


(Jl) 




Then we have for A £ O : 
(a) 

ch. 




/^eo 

~ tw,H 

ctlA 




g-2ui(fc|«|2+A(0) 




(b) 


chx 


ill. 


= e^-^cht, chT^ 


— g27ri(sA+A(0)g’r*fc|?l ch 


2 "Y tw,=f 


(Jl) 


□ 
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